• 沒有找到結果。

# Solutions of Equations in One Variable

N/A
N/A
Protected

Share "Solutions of Equations in One Variable"

Copied!
68
0
0

(1)

Tsung-Ming Huang

Department of Mathematics National Taiwan Normal University, Taiwan

August 28, 2011

(2)

### Outline

1 Bisection Method

2 Fixed-Point Iteration

3 Newton’s method

4 Error analysis for iterative methods

5 Accelerating convergence

6 Zeros of polynomials and M ¨uller’s method

(3)

### Bisection Method

Idea

Iff (x) ∈ C[a, b]andf (a)f (b) < 0, then∃ c ∈ (a, b)such that f (c) = 0.

(4)

Bisection method algorithm

Given f (x) defined on (a, b), the maximal number of iterations M, and stop criteria δ and ε, this algorithm tries to locate one root of f (x).

Compute u = f (a), v = f (b), and e = b − a If sign(u) = sign(v), then stop

For k = 1, 2, . . . , M

e = e/2, c = a + e, w = f (c) If |e| < δ or |w| < ε, then stop If sign(w) 6= sign(u)

b = c, v = w Else

a = c, u = w End If

End For

(5)

Let {cn} be the sequence of numbers produced. The algorithm should stop if one of the following conditions is satisfied.

1 the iteration number k > M ,

2 |ck− ck−1| < δ, or

3 |f (ck)| < ε.

Let [a0, b0], [a1, b1], . . .denote the successive intervals produced by the bisection algorithm. Then

a = a0≤ a1 ≤ a2 ≤ · · · ≤ b0= b

⇒ {an} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

(6)

Since

b1− a1 = 1

2(b0− a0) b2− a2 = 1

2(b1− a1) = 1

4(b0− a0) ...

bn− an = 1

2n(b0− a0) hence

n→∞lim bn− lim

n→∞an= lim

n→∞(bn− an) = lim

n→∞

1

2n(b0− a0) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

(7)

On the other hand,

f (an)f (bn) ≤ 0

⇒ lim

n→∞f (an)f (bn) = f2(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {an} and {bn} is a zero of f in [a, b]. Let cn= 12(an+ bn). Then

|z − cn| = lim

n→∞an−1

2(an+ bn)

= 1 2

h

n→∞lim an− bni + 1

2 h

n→∞lim an− ani

≤ max lim

n→∞an− bn ,

lim

n→∞an− an

≤ |bn− an| = 1

2n|b0− a0|.

This proves the following theorem.

(8)

Theorem 1

Let{[an, bn]}denote the intervals produced by the bisection algorithm. Then lim

n→∞anand lim

n→∞bnexist, areequal, and represent azerooff (x). If

z = lim

n→∞an= lim

n→∞bn and cn= 1

2(an+ bn), then

|z − cn| ≤ 1

2n(b0− a0) . Remark

{cn}convergestozwith therateofO(2−n).

(9)

Example 2

How many steps should be taken to compute a root of f (x) = x3+ 4x2− 10 = 0 on [1, 2] with relative error 10−3? solution: Seek an n such that

|z − cn|

|z| ≤ 10−3 ⇒ |z − cn| ≤ |z| × 10−3. Since z ∈ [1, 2], it is sufficient to show

|z − cn| ≤ 10−3. That is, we solve

2−n(2 − 1) ≤ 10−3 ⇒ −n log102 ≤ −3 which gives n ≥ 10.

(10)

### Fixed-Point Iteration

Definition 3

xis called afixed pointof a given functionf iff (x) = x.

Root-finding problems and fixed-point problems Find x such that f (x) = 0.

Let g(x) = x − f (x). Then g(x) = x− f (x) = x.

⇒ xis a fixed point for g(x).

Find x such that g(x) = x. Define f (x) = x − g(x) so that f (x) = x− g(x) = x− x= 0

⇒ xis a zero of f (x).

(11)

Example 4

The function g(x) = x2− 2, for −2 ≤ x ≤ 3, has fixed points at x = −1and x = 2 since

g(−1) = (−1)2− 2 = −1 and g(2) = 22− 2 = 2.

(12)

Theorem 5 (Existence and uniqueness)

1 If g ∈ C[a, b] such thata ≤ g(x) ≤ bfor all x ∈ [a, b], then g hasa fixed point in [a, b].

2 If, in addition, g0(x)exists in (a, b) and there exists a positive constantM < 1such that|g0(x)| ≤ M < 1for all x ∈ (a, b). Then the fixed point isunique.

(13)

### Proof

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x ∈ [a, b] such that h(x) = 0. That is

g(x) − x = 0 ⇒ g(x) = x. Hence g has a fixed point xin [a, b].

(14)

### Proof

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b]. By the Mean-Value theorem, there exists ξ between p and q such that

g0(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g0(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is unique.

(15)

Example 6

Show that the following function has a unique fixed point.

g(x) = (x2− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g0(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

(16)

Let p be such unique fixed point of g. Then p = g(p) = p2− 1

3 ⇒ p2− 3p − 1 = 0

⇒ p = 1 2(3 −√

13).

(17)

Fixed-point iteration or functional iteration

Given a continuous function g, choose an initial point x0 and generate {xk}k=0by

xk+1 = g(xk), k ≥ 0.

{xk} may not converge, e.g., g(x) = 3x. However, when the sequence converges, say,

k→∞lim xk= x, then, since g is continuous,

g(x) = g( lim

k→∞xk) = lim

k→∞g(xk) = lim

k→∞xk+1= x. That is, xis a fixed point of g.

(18)

Fixed-point iteration

Given x0, tolerance T OL, maximum number of iteration M . Set i = 1 and x = g(x0).

While i ≤ M and |x − x0| ≥ T OL Set i = i + 1, x0 = xand x = g(x0).

End While

(19)

Example 7 The equation

x3+ 4x2− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g1(x) ≡ x − f (x) = x − x3− 4x2+ 10

(b) x = g2(x) = 10x − 4x1/2

x3 = 10 − 4x2 ⇒ x2 = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

(20)

(c) x = g3(x) = 12 10 − x31/2

4x2 = 10 − x3 ⇒ x = ±1

2 10 − x31/2

(d) x = g4(x) =

10 4+x

1/2

x2(x + 4) = 10 ⇒ x = ±

 10 4 + x

1/2

(e) x = g5(x) = x −x33x+4x2+8x2−10

x = g5(x) ≡ x − f (x) f0(x)

(21)

Results of the fixed-point iteration with initial point x0 = 1.5

(22)

Theorem 8 (Fixed-point Theorem)

Let g ∈ [a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b]. Suppose that g0exists on (a, b) and that ∃ k with0 < k < 1such that

|g0(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x0 in [a, b],

xn= g(xn−1), n ≥ 1, converges to the unique fixed point x in [a, b].

(23)

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {xn}n=0 is defined and xn∈ [a, b] for all n ≥ 0. Using the Mean Values Theorem and the fact that

|g0(x)| ≤ k, we have

|x − xn| = |g(xn−1) − g(x)| = |g0n)||x − xn−1| ≤ k|x − xn−1|, where ξn∈ (a, b). It follows that

|xn− x| ≤ k|xn−1− x| ≤ k2|xn−2− x| ≤ · · · ≤ kn|x0− x|. (1) Since 0 < k < 1, we have

n→∞lim kn= 0 and

n→∞lim |xn− x| ≤ lim

n→∞kn|x0− x| = 0.

Hence, {xn}n=0converges to x.

(24)

Corollary 9

If g satisfies the hypotheses of above theorem, then

|x − xn| ≤ knmax{x0− a, b − x0} and

|xn− x| ≤ kn

1 − k|x1− x0|, ∀ n ≥ 1.

Proof: From (1),

|xn− x| ≤ kn|x0− x| ≤ knmax{x0− a, b − x0}.

For n ≥ 1, using the Mean Values Theorem,

|xn+1− xn| = |g(xn) − g(xn−1)| ≤ k|xn− xn−1| ≤ · · · ≤ kn|x1− x0|.

(25)

Thus, for m > n ≥ 1,

|xm− xn| = |xm− xm−1+ xm−1− · · · + xn+1− xn|

≤ |xm− xm−1| + |xm−1− xm−2| + · · · + |xn+1− xn|

≤ km−1|x1− x0| + km−2|x1− x0| + · · · + kn|x1− x0|

= kn|x1− x0| 1 + k + k2+ · · · + km−n−1 . It implies that

|x − xn| = lim

m→∞|xm− xn| ≤ lim

m→∞kn|x1− x0|

m−n−1

X

j=0

kj

≤ kn|x1− x0|

X

j=0

kj = kn

1 − k|x1− x0|.

(26)

Example 10

For previous example,

f (x) = x3+ 4x2− 10 = 0.

For g1(x) = x − x3− 4x2+ 10, we have g1(1) = 6 and g1(2) = −12, so g1([1, 2]) * [1, 2]. Moreover,

g10(x) = 1 − 3x2− 8x ⇒ |g10(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

(27)

For g3(x) = 12(10 − x3)1/2, ∀ x ∈ [1, 1.5], g30(x) = −3

4x2(10 − x3)−1/2< 0, ∀ x ∈ [1, 1.5], so g3 is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g3(1.5) ≤ g3(x) ≤ g3(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g03(x)| ≤ |g03(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

(28)

For g4(x) =p10/(4 + x), we have r10

6 ≤ g4(x) ≤ r10

5 , ∀ x ∈ [1, 2] ⇒ g4([1, 2]) ⊆ [1, 2]

Moreover,

|g04(x)| =

√ −5

10(4 + x)3/2

≤ 5

√10(5)3/2 < 0.15, ∀ x ∈ [1, 2].

The bound of |g04(x)|is much smaller than the bound of |g30(x)|, which explains the more rapid convergence using g4.

(29)

Suppose that f : R → R and f ∈ C2[a, b], i.e., f00exists and is continuous. If f (x) = 0and x= x + hwhere h is small, then byTaylor’stheorem

0 = f (x) = f (x + h)

= f (x) + f0(x)h + 1

2f00(x)h2+ 1

3!f000(x)h3+ · · ·

= f (x) + f0(x)h + O(h2).

Sincehissmall, O(h2)is negligible. It is reasonable to drop O(h2)terms. This implies

f (x) + f0(x)h ≈ 0 and h ≈ −f (x)

f0(x), if f0(x) 6= 0.

Hence

x + h = x − f (x) f0(x) is a better approximation to x.

(30)

This sets the stage for theNewton-Rapbson’smethod, which starts with an initial approximation x0 and generates the sequence {xn}n=0defined by

xn+1 = xn− f (xn) f0(xn).

Since the Taylor’s expansion of f (x) at xkis given by f (x) = f (xk) + f0(xk)(x − xk) +1

2f00(xk)(x − xk)2+ · · · . At xk, one uses thetangent line

y = `(x) = f (xk) + f0(xk)(x − xk)

toapproximate the curveof f (x) and uses the zero of the tangent line to approximate the zero of f (x).

(31)

Newton’s Method

Given x0, tolerance T OL, maximum number of iteration M . Set i = 1 and x = x0− f (x0)/f0(x0).

While i ≤ M and |x − x0| ≥ T OL

Set i = i + 1, x0 = xand x = x0− f (x0)/f0(x0).

End While

(32)

Three stopping-technique inequalities

(a). |xn− xn−1| < ε, (b). |xn− xn−1|

|xn| < ε, xn6= 0, (c). |f (xn)| < ε.

Note that Newton’s method for solving f (x) = 0 xn+1= xn− f (xn)

f0(xn), for n ≥ 1 is just a special case of functional iteration in which

g(x) = x − f (x) f0(x).

(33)

Example 11

The following table shows the convergence behavior of

Newton’s method applied to solving f (x) = x2− 1 = 0. Observe the quadratic convergence rate.

n xn |en| ≡ |1 − xn|

0 2.0 1

1 1.25 0.25

2 1.025 2.5e-2

3 1.0003048780488 3.048780488e-4 4 1.0000000464611 4.64611e-8

5 1.0 0

(34)

Theorem 12

Assumef (x) = 0, f0(x) 6= 0andf (x),f0(x)andf00(x)are continuouson Nε(x). Then if x0is chosensufficiently closeto x, then



xn+1 = xn− f (xn) f0(xn)



→ x.

Proof: Define

g(x) = x − f (x) f0(x). Find an interval [x− δ, x+ δ]such that

g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ]

and

|g0(x)| ≤ k < 1, ∀ x ∈ (x− δ, x+ δ).

(35)

Since f0 is continuous and f0(x) 6= 0, it implies that ∃ δ1 > 0 such that f0(x) 6= 0 ∀ x ∈ [x− δ1, x+ δ1] ⊆ [a, b]. Thus, g is defined and continuous on [x− δ1, x+ δ1]. Also

g0(x) = 1 − f0(x)f0(x) − f (x)f00(x)

[f0(x)]2 = f (x)f00(x) [f0(x)]2 , for x ∈ [x− δ1, x+ δ1]. Since f00 is continuous on [a, b], we have g0 is continuous on [x− δ1, x+ δ1].

By assumption f (x) = 0, so

g0(x) = f (x)f00(x)

|f0(x)|2 = 0.

Since g0 is continuous on [x− δ1, x+ δ1]and g0(x) = 0, ∃ δ with 0 < δ < δ1 and k ∈ (0, 1) such that

|g0(x)| ≤ k, ∀ x ∈ [x− δ, x+ δ].

(36)

Claim: g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ].

If x ∈ [x− δ, x+ δ], then, by the Mean Value Theorem, ∃ ξ between x and x such that

|g(x) − g(x)| = |g0(ξ)||x − x|.

It implies that

|g(x) − x| = |g(x) − g(x)| = |g0(ξ)||x − x|

≤ k|x − x| < |x − x| < δ.

Hence, g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ].

By the Fixed-Point Theorem, the sequence {xn}n=0defined by xn= g(xn−1) = xn−1− f (xn−1)

f0(xn−1), for n ≥ 1, converges to x for any x0 ∈ [x− δ, x+ δ].

(37)

Example 13

When Newton’s method applied to f (x) = cos x with starting point x0 = 3, which is close to the root π2 of f , it produces x1 = −4.01525, x2= −4.8526, · · · ,which converges to another root −2 .

5 4 3 2 1 0 1 2 3 4 5

1. 5 0 1.5

x0 y = c os ( x)

x*

(38)

### Secant method

In many applications, the derivative f0(x)is very expensive to compute, or the function f (x) is not given in an algebraic formula so that f0(x)is not available.

By definition,

f0(xn−1) = lim

x→xn−1

f (x) − f (xn−1) x − xn−1 . Letting x = xn−2, we have

f0(xn−1) ≈ f (xn−2) − f (xn−1)

xn−2− xn−1 = f (xn−1) − f (xn−2) xn−1− xn−2 . Using this approximation for f0(xn−1)in Newton’s formula gives

xn= xn−1−f (xn−1)(xn−1− xn−2) f (xn−1) − f (xn−2) ,

38 / 68

(39)

From geometric point of view, we use asecant linethrough xn−1andxn−2 instead of the tangent line to approximate the function at the point xn−1.

The slope of the secant line is

sn−1 = f (xn−1) − f (xn−2) xn−1− xn−2 and the equation is

M (x) = f (xn−1) + sn−1(x − xn−1).

The zero of the secant line x = xn−1−f (xn−1)

sn−1 = xn−1− f (xn−1) xn−1− xn−2 f (xn−1) − f (xn−2) is then used as a new approximate xn.

(40)

Secant Method

Given x0, x1, tolerance T OL, maximum number of iteration M . Set i = 2; y0 = f (x0); y1= f (x1);

x = x1− y1(x1− x0)/(y1− y0).

While i ≤ M and |x − x1| ≥ T OL

Set i = i + 1; x0 = x1; y0 = y1; x1= x; y1 = f (x);

x = x1− y1(x1− x0)/(y1− y0).

End While

(41)

Method of False Position

1 Choose initial approximations x0 and x1 with f (x0)f (x1) < 0.

2 x2= x1− f (x1)(x1− x0)/(f (x1) − f (x0))

3 Decide which secant line to use to compute x3: If f (x2)f (x1) < 0, then x1and x2 bracket a root, i.e.,

x3= x2− f (x2)(x2− x1)/(f (x2) − f (x1)) Else, x0 and x2 bracket a root, i.e.,

x3= x2− f (x2)(x2− x0)/(f (x2) − f (x0)) End if

(42)

Method of False Position

Given x0, x1, tolerance T OL, maximum number of iteration M . Set i = 2; y0 = f (x0); y1= f (x1); x = x1− y1(x1− x0)/(y1− y0).

While i ≤ M and |x − x1| ≥ T OL Set i = i + 1; y = f (x).

If y · y1< 0, then set x0= x1; y0 = y1.

Set x1 = x; y1 = y; x = x1− y1(x1− x0)/(y1− y0).

End While

(43)

### Error analysis for iterative methods

Definition 14

Let{xn} → x. If there are positive constantscandαsuch that

n→∞lim

|xn+1− x|

|xn− x|α = c,

then we say therate of convergenceis oforderα.

We say that the rate of convergence is

1 linearifα = 1and 0 < c < 1.

2 superlinearif

n→∞lim

|xn+1− x|

|xn− x| = 0;

3 quadraticifα = 2. 43 / 68

(44)

Suppose that {xn}n=0and {˜xn}n=0are linearly and quadratically convergent to x, respectively, with the same constant c = 0.5. For simplicity, suppose that

|xn+1− x|

|xn− x| ≈ c and |˜xn+1− x|

|˜xn− x|2 ≈ c.

These imply that

|xn− x| ≈ c|xn−1− x| ≈ c2|xn−2− x| ≈ · · · ≈ cn|x0− x|, and

|˜xn− x| ≈ c|˜xn−1− x|2 ≈ cc|˜xn−2− x|22

= c3|˜xn−2− x|4

≈ c3c|˜xn−3− x|24

= c7|˜xn−3− x|8

≈ · · · ≈ c2n−1|˜x0− x|2n.

(45)

Remark

Quadratically convergent sequences generally converge much more quickly thank those that converge only linearly.

Theorem 15

Let g ∈ C[a, b] with g([a, b]) ⊆ [a, b]. Suppose that g0 is continuous on (a, b) and ∃ k ∈ (0, 1) such that

|g0(x)| ≤ k, ∀ x ∈ (a, b).

Ifg0(x) 6= 0, then for any x0∈ [a, b], the sequence xn= g(xn−1), for n ≥ 1

converges onlylinearlyto the unique fixed point x in [a, b].

(46)

Proof:

By the Fixed-Point Theorem, the sequence {xn}n=0 converges to x.

Since g0 exists on (a, b), by the Mean Value Theorem, ∃ ξn between xnand x such that

xn+1− x = g(xn) − g(x) = g0n)(xn− x).

∵ {xn}n=0→ x ⇒ {ξn}n=0→ x Since g0 is continuous on (a, b), we have

n→∞lim g0n) = g0(x).

Thus,

n→∞lim

|xn+1− x|

|xn− x| = lim

n→∞|g0n)| = |g0(x)|.

Hence, if g0(x) 6= 0, fixed-point iteration exhibits linear convergence.

(47)

Theorem 16

Let x be a fixed point of g and I be an open interval with x ∈ I. Suppose thatg0(x) = 0and g00is continuous with

|g00(x)| < M, ∀ x ∈ I.

Then ∃ δ > 0 such that

{xn= g(xn−1)}n=1 → x for x0 ∈ [x− δ, x+ δ]

|xn+1− x| < M

2 |xn− x|2, for sufficiently large n.

(48)

Proof:

Since g0(x) = 0and g0 is continuous on I, ∃ δ such that [x− δ, x+ δ] ⊂ Iand

|g0(x)| ≤ k < 1, ∀ x ∈ [x− δ, x+ δ].

In the proof of the convergence for Newton’s method, we have

{xn}n=0⊂ [x− δ, x+ δ].

Consider the Taylor expansion of g(xn)at x xn+1= g(xn) = g(x) + g0(x)(xn− x) +g00(ξ)

2 (xn− x)2

= x+g00(ξ)

2 (xn− x)2, where ξ lies between xnand x.

(49)

Since

|g0(x)| ≤ k < 1, ∀ x ∈ [x− δ, x+ δ]

and

g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ], it follows that {xn}n=0converges to x.

But ξnis between xnand xfor each n, so {ξn}n=0also converges to x and

n→∞lim

|xn+1− x|

|xn− x|2 = |g00(x)|

2 < M 2 .

It implies that {xn}n=0 is quadratically convergent to xif g00(x) 6= 0and

|xn+1− x| < M

2 |xn− x|2, for sufficiently large n.

(50)

For Newton’s method, g(x) = x − f (x)

f0(x) ⇒ g0(x) = 1 − f0(x)

f0(x)+f (x)f00(x)

(f0(x))2 = f (x)f00(x) (f0(x))2 It follows that g0(x) = 0. Hence Newton’s method is locally quadratically convergent.

(51)

### Error Analysis of Secant Method

Reference: D. Kincaid and W. Cheney, ”Numerical analysis”

Let x denote the exact solution of f (x) = 0, ek= xk− x be the error at the k-th step. Then

ek+1 = xk+1− x

= xk− f (xk) xk− xk−1

f (xk) − f (xk−1) − x

= 1

f (xk) − f (xk−1)[(xk−1− x)f (xk) − (xk− x)f (xk−1)]

= 1

f (xk) − f (xk−1)(ek−1f (xk) − ekf (xk−1))

= ekek−1

1

ekf (xk) − e1

k−1f (xk−1)

xk− xk−1 · xk− xk−1 f (xk) − f (xk−1)

!

(52)

To estimate the numerator

1

ekf (xk)− 1

ek−1f (xk−1)

xk−xk−1 , we apply Taylor’s Theorem

f (xk) = f (x+ ek) = f (x) + f0(x)ek+1

2f00(x)e2k+ O(e3k), to get

1 ek

f (xk) = f0(x) +1

2f00(x)ek+ O(e2k).

Similarly, 1

ek−1f (xk−1) = f0(x) +1

2f00(x)ek−1+ O(e2k−1).

Hence 1

ekf (xk) − 1

ek−1f (xk−1) ≈ 1

2(ek− ek−1)f00(x).

Since xk− xk−1 = ek− ek−1and xk− xk−1

f (x ) − f (x ) → 1 f0(x),

(53)

we have

ek+1 ≈ ekek−1

1

2(ek− ek−1)f00(x) ek− ek−1 · 1

f0(x)

!

= 1 2

f00(x) f0(x)ekek−1

≡ Cekek−1. (2)

To estimate the convergence rate, we assume

|ek+1| ≈ η|ek|α, where η > 0 and α > 0 are constants, i.e.,

|ek+1|

η|ek|α → 1 as k → ∞.

Then |ek| ≈ η|ek−1|αwhich implies |ek−1| ≈ η−1/α|ek|1/α. Hence (2) gives

η|ek|α≈ C|ek−1/α|ek|1/α =⇒ C−1η1+α1 ≈ |ek|1−α+α1.

(54)

Since |ek| → 0 as k → ∞, and C−1η1+α1 is a nonzero constant, 1 − α + 1

α = 0 =⇒ α = 1 +√ 5

2 ≈ 1.62.

This result implies that C−1η1+α1 → 1 and η → C1+αα = f00(x)

2f0(x)

0.62

. In summary, we have shown that

|ek+1| = η|ek|α, α ≈ 1.62, that is, therate of convergenceissuperlinear.

Rate of convergence

secantmethod: superlinear Newton’smethod: quadratic bisectionmethod: linear

(55)

Each iteration of method requires

secant method: one function evaluation

Newton’s method: two function evaluation, namely, f (xk) and f0(xk).

⇒ two steps of secant method are comparable to one step of Newton’s method. Thus

|ek+2| ≈ η|ek+1|α ≈ η1+α|ek|3+

5

2 ≈ η1+α|ek|2.62.

⇒ secant method is more efficient than Newton’s method.

Remark

Two steps of secant method would require a little more work than one step of Newton’s method.

(56)

Aitken’s ∆2 method

Accelerate the convergence of a sequence that islinearly convergent.

Suppose {yn}n=0is a linearly convergent sequence with limit y. Construct a sequence {ˆyn}n=0that converges more rapidly to y than {yn}n=0.

For n sufficiently large, yn+1− y

yn− y ≈ yn+2− y yn+1− y. Then

(yn+1− y)2≈ (yn+2− y)(yn− y), so

y2n+1− 2yn+1y + y2 ≈ yn+2yn− (yn+2+ yn)y + y2

(57)

and

(yn+2+ yn− 2yn+1)y ≈ yn+2yn− y2n+1. Solving for y gives

y ≈ yn+2yn− yn+12 yn+2− 2yn+1+ yn

= ynyn+2− 2ynyn+1+ y2n− yn2+ 2ynyn+1− y2n+1 yn+2− 2yn+1+ yn

= yn(yn+2− 2yn+1+ yn) − (yn+1− yn)2 (yn+2− yn+1) − (yn+1− yn)

= yn− (yn+1− yn)2

(yn+2− yn+1) − (yn+1− yn). Aitken’s ∆2 method

ˆ

yn= yn− (yn+1− yn)2

(yn+2− yn+1) − (yn+1− yn). (3)

(58)

Example 17

The sequence {yn= cos(1/n)}n=1converges linearly to y = 1.

n ynn

1 0.54030 0.96178 2 0.87758 0.98213 3 0.94496 0.98979 4 0.96891 0.99342 5 0.98007 0.99541 6 0.98614

7 0.98981

{ˆyn}n=1converges more rapidly to y = 1 than {yn}n=1.

(59)

Definition 18

For a given sequence {yn}n=0, the forward difference ∆ynis defined by

∆yn= yn+1− yn, for n ≥ 0.

Higher powers of ∆ are defined recursively by

kyn= ∆(∆k−1yn), for k ≥ 2.

The definition implies that

2yn= ∆(yn+1− yn) = ∆yn+1− ∆yn= (yn+2− yn+1) − (yn+1− yn).

So the formula for ˆynin (3) can be written as ˆ

yn= yn−(∆yn)2

2yn

, for n ≥ 0.

(60)

Theorem 19

Suppose {yn}n=0 → ylinearlyand

n→∞lim

yn+1− y yn− y < 1.

Then {ˆyn}n=0→ y faster than {yn}n=0in the sense that

n→∞lim ˆ yn− y yn− y = 0.

Aitken’s ∆2method constructs the terms in order:

y0, y1 = g(y0), y2= g(y1), yˆ0= {∆2}(y0), y3 = g(y2), ˆ

y1 = {∆2}(y1), . . . .

⇒ Assume |ˆy0− y| < |y2− y|

(61)

Steffensen’s method constructs the terms in order:

y0(0)≡ y0, y1(0) = g(y(0)0 ), y2(0)= g(y1(0)), y0(1)= {∆2}(y0(0)), y(1)1 = g(y(1)0 ), y2(1)= g(y(1)1 ), . . . . Steffensen’s method (To find a solution of y = g(y))

Given y0, tolerance T ol, max. number of iteration M . Set i = 1.

While i ≤ M

Set y1= g(y0); y2= g(y1); y = y0− (y1− y0)2/(y2− 2y1+ y0).

If |y − y0| < T ol, then STOP.

Set i = i + 1; y0 = y.

End While Theorem 20

Suppose x = g(x) has solution x with g0(x) 6= 1. If ∃ δ > 0 such that g ∈ C3[x− δ, x+ δ], then Steffensen’s method gives quadraticconvergence for any x0 ∈ [x− δ, x+ δ].

(62)

### Zeros of polynomials and M ¨uller’s method

• Horner’s method:

Let

P (x) = a0+ a1x + a2x2+ · · · + an−1xn−1+ anxn

= a0+ x (a1+ x (a2+ · · · + x (an−1+ anx) · · · )) . If

bn = an,

bk = ak+ bk+1x0, for k = n − 1, n − 2, . . . , 1, 0, then

b0= a0+ b1x0= a0+ (a1+ b2x0) x0 = · · · = P (x0).

Consider

Q(x) = b1+ b2x + · · · + bnxn−1.

(63)

Then

b0+ (x − x0)Q(x) = b0+ (x − x0) b1+ b2x + · · · + bnxn−1

= (b0− b1x0) + (b1− b2x0)x + · · · + (bn−1− bnx0)xn−1+ bnxn

= a0+ a1x + · · · + anxn= P (x).

Differentiating P (x) with respect to x gives

P0(x) = Q(x) + (x − x0)Q0(x) and P0(x0) = Q(x0).

Use Newton-Raphson method to find an approximate zero of P (x):

xk+1= xk−P (xk)

Q(xk), ∀ k = 0, 1, 2, . . . . Similarly, let

cn = bn= an,

ck = bk+ ck+1xk, for k = n − 1, n − 2, . . . , 1, then c1 = Q(xk).

(64)

Horner’s method(Evaluate y = P (x0)and z = P0(x0)) Set y = an; z = an.

For j = n − 1, n − 2, . . . , 1 Set y = aj+ yx0; z = y + zx0. End for

Set y = a0+ yx0. If xN is an approximate zero of P , then

P (x) = (x − xN)Q(x) + b0 = (x − xN)Q(x) + P (xN)

≈ (x − xN)Q(x) ≡ (x − ˆx1)Q1(x).

So x − ˆx1is an approximate factor of P (x) and we can find a second approximate zero of P by applying Newton’s method to Q1(x). The procedure is called deflation.

(65)

• M ¨uller’s method for complex root:

Theorem 21

If z = a + ib is a complex zero of multiplicity m of P (x) with real coefficients, then ¯z = a − biis also a zero of multiplicity m of P (x)and (x2− 2ax + a2+ b2)m is a factor of P (x).

Secant method: Given p0 and p1, determine p2 as the intersection of the x-axis with the line through (p0, f (p0))and (p1, f (p1)).

M ¨uller’s method: Given p0, p1

and p2, determine p3 by the intersection of the x-axis with the parabola through (p0, f (p0)), (p1, f (p1))and (p2, f (p2)).

(66)

Let

P (x) = a(x − p2)2+ b(x − p2) + c

that passes through (p0, f (p0)), (p1, f (p1))and (p2, f (p2)). Then f (p0) = a(p0− p2)2+ b(p0− p2) + c,

f (p1) = a(p1− p2)2+ b(p1− p2) + c, f (p2) = a(p2− p2)2+ b(p2− p2) + c = c.

It implies that c = f (p2),

b = (p0− p2)2[f (p1) − f (p2)] − (p1− p2)2[f (p0) − f (p2)]

(p0− p2)(p1− p2)(p0− p1) , a = (p1− p2) [f (p0) − f (p2)] − (p0− p2) [f (p1) − f (p2)]

(p0− p2)(p1− p2)(p0− p1) .

(67)

To determine p3, a zero of P , we apply the quadratic formula to P (x) = 0and get

p3− p2 = 2c b ±√

b2− 4ac. Choose

p3 = p2+ 2c b + sgn(b)√

b2− 4ac

such that the denominator will be largest and result in p3 selected as the closest zero of P to p2.

(68)

M ¨uller’s method (Find a solution of f (x) = 0)

Given p0, p1, p2; tolerance T OL; maximum number of iterations M Set h1 = p1− p0; h2 = p2− p1;

δ1= (f (p1) − f (p0))/h1; δ2 = (f (p2) − f (p1))/h2; d = (δ2− δ1)/(h2+ h1); i = 3.

While i ≤ M

Set b = δ2+ h2d; D =pb2− 4f (p2)d.

If |b − D| < |b + D|, then set E = b + D else set E = b − D.

Set h = −2f (p2)/E; p = p2+ h.

If |h| < T OL, then STOP.

Set p0= p1; p1 = p2; p2 = p; h1= p1− p0; h2= p2− p1; δ1= (f (p1) − f (p0))/h1; δ2 = (f (p2) − f (p1))/h2; d = (δ2− δ1)/(h2+ h1); i = i + 1.

End while

1. In the textbook, pp 224-223, there is a surprising result called Space- filling curve. In addition, note the proof is related with Cantor set in exercise 7. There exists a

The method of least squares is a standard approach to the approximate solution of overdetermined systems, i.e., sets of equations in which there are more equations than unknowns.

 Combine: find closet pair with one point in each region, and return the best of three

Quantitative uniqueness of solutions to second order elliptic equations with singular lower order terms.. Quantitative uniqueness of solutions to second order elliptic equations

Theorem 1.5.9 ﬁnite dimensional vector space linearly independent set 大 basis; spanning

“Find sufficiently accurate starting approximate solution by using Steepest Descent method” + ”Compute convergent solution by using Newton-based methods”. The method of

denote the successive intervals produced by the bisection algorithm... denote the successive intervals produced by the

Other researchers say one way to solve the problem of wasted food is to take steps to persuade people to stop buying so much food in the first place.. People buy more food