# Solutions of Equations in One Variable

## Full text

(1)

Tsung-Ming Huang

Department of Mathematics National Taiwan Normal University, Taiwan

August 28, 2011

(2)

### Outline

1 Bisection Method

2 Fixed-Point Iteration

3 Newton’s method

4 Error analysis for iterative methods

5 Accelerating convergence

6 Zeros of polynomials and M ¨uller’s method

(3)

### Bisection Method

Idea

Iff (x) ∈ C[a, b]andf (a)f (b) < 0, then∃ c ∈ (a, b)such that f (c) = 0.

(4)

Bisection method algorithm

Given f (x) defined on (a, b), the maximal number of iterations M, and stop criteria δ and ε, this algorithm tries to locate one root of f (x).

Compute u = f (a), v = f (b), and e = b − a If sign(u) = sign(v), then stop

For k = 1, 2, . . . , M

e = e/2, c = a + e, w = f (c) If |e| < δ or |w| < ε, then stop If sign(w) 6= sign(u)

b = c, v = w Else

a = c, u = w End If

End For

(5)

Let {cn} be the sequence of numbers produced. The algorithm should stop if one of the following conditions is satisfied.

1 the iteration number k > M ,

2 |ck− ck−1| < δ, or

3 |f (ck)| < ε.

Let [a0, b0], [a1, b1], . . .denote the successive intervals produced by the bisection algorithm. Then

a = a0≤ a1 ≤ a2 ≤ · · · ≤ b0= b

⇒ {an} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

(6)

Since

b1− a1 = 1

2(b0− a0) b2− a2 = 1

2(b1− a1) = 1

4(b0− a0) ...

bn− an = 1

2n(b0− a0) hence

n→∞lim bn− lim

n→∞an= lim

n→∞(bn− an) = lim

n→∞

1

2n(b0− a0) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

(7)

On the other hand,

f (an)f (bn) ≤ 0

⇒ lim

n→∞f (an)f (bn) = f2(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {an} and {bn} is a zero of f in [a, b]. Let cn= 12(an+ bn). Then

|z − cn| = lim

n→∞an−1

2(an+ bn)

= 1 2

h

n→∞lim an− bni + 1

2 h

n→∞lim an− ani

≤ max lim

n→∞an− bn ,

lim

n→∞an− an

≤ |bn− an| = 1

2n|b0− a0|.

This proves the following theorem.

(8)

Theorem 1

Let{[an, bn]}denote the intervals produced by the bisection algorithm. Then lim

n→∞anand lim

n→∞bnexist, areequal, and represent azerooff (x). If

z = lim

n→∞an= lim

n→∞bn and cn= 1

2(an+ bn), then

|z − cn| ≤ 1

2n(b0− a0) . Remark

{cn}convergestozwith therateofO(2−n).

(9)

Example 2

How many steps should be taken to compute a root of f (x) = x3+ 4x2− 10 = 0 on [1, 2] with relative error 10−3? solution: Seek an n such that

|z − cn|

|z| ≤ 10−3 ⇒ |z − cn| ≤ |z| × 10−3. Since z ∈ [1, 2], it is sufficient to show

|z − cn| ≤ 10−3. That is, we solve

2−n(2 − 1) ≤ 10−3 ⇒ −n log102 ≤ −3 which gives n ≥ 10.

(10)

### Fixed-Point Iteration

Definition 3

xis called afixed pointof a given functionf iff (x) = x.

Root-finding problems and fixed-point problems Find x such that f (x) = 0.

Let g(x) = x − f (x). Then g(x) = x− f (x) = x.

⇒ xis a fixed point for g(x).

Find x such that g(x) = x. Define f (x) = x − g(x) so that f (x) = x− g(x) = x− x= 0

⇒ xis a zero of f (x).

(11)

Example 4

The function g(x) = x2− 2, for −2 ≤ x ≤ 3, has fixed points at x = −1and x = 2 since

g(−1) = (−1)2− 2 = −1 and g(2) = 22− 2 = 2.

(12)

Theorem 5 (Existence and uniqueness)

1 If g ∈ C[a, b] such thata ≤ g(x) ≤ bfor all x ∈ [a, b], then g hasa fixed point in [a, b].

2 If, in addition, g0(x)exists in (a, b) and there exists a positive constantM < 1such that|g0(x)| ≤ M < 1for all x ∈ (a, b). Then the fixed point isunique.

(13)

### Proof

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x ∈ [a, b] such that h(x) = 0. That is

g(x) − x = 0 ⇒ g(x) = x. Hence g has a fixed point xin [a, b].

(14)

### Proof

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b]. By the Mean-Value theorem, there exists ξ between p and q such that

g0(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g0(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is unique.

(15)

Example 6

Show that the following function has a unique fixed point.

g(x) = (x2− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g0(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

(16)

Let p be such unique fixed point of g. Then p = g(p) = p2− 1

3 ⇒ p2− 3p − 1 = 0

⇒ p = 1 2(3 −√

13).

(17)

Fixed-point iteration or functional iteration

Given a continuous function g, choose an initial point x0 and generate {xk}k=0by

xk+1 = g(xk), k ≥ 0.

{xk} may not converge, e.g., g(x) = 3x. However, when the sequence converges, say,

k→∞lim xk= x, then, since g is continuous,

g(x) = g( lim

k→∞xk) = lim

k→∞g(xk) = lim

k→∞xk+1= x. That is, xis a fixed point of g.

(18)

Fixed-point iteration

Given x0, tolerance T OL, maximum number of iteration M . Set i = 1 and x = g(x0).

While i ≤ M and |x − x0| ≥ T OL Set i = i + 1, x0 = xand x = g(x0).

End While

(19)

Example 7 The equation

x3+ 4x2− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g1(x) ≡ x − f (x) = x − x3− 4x2+ 10

(b) x = g2(x) = 10x − 4x1/2

x3 = 10 − 4x2 ⇒ x2 = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

(20)

(c) x = g3(x) = 12 10 − x31/2

4x2 = 10 − x3 ⇒ x = ±1

2 10 − x31/2

(d) x = g4(x) =

10 4+x

1/2

x2(x + 4) = 10 ⇒ x = ±

 10 4 + x

1/2

(e) x = g5(x) = x −x33x+4x2+8x2−10

x = g5(x) ≡ x − f (x) f0(x)

(21)

Results of the fixed-point iteration with initial point x0 = 1.5

(22)

Theorem 8 (Fixed-point Theorem)

Let g ∈ [a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b]. Suppose that g0exists on (a, b) and that ∃ k with0 < k < 1such that

|g0(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x0 in [a, b],

xn= g(xn−1), n ≥ 1, converges to the unique fixed point x in [a, b].

(23)

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {xn}n=0 is defined and xn∈ [a, b] for all n ≥ 0. Using the Mean Values Theorem and the fact that

|g0(x)| ≤ k, we have

|x − xn| = |g(xn−1) − g(x)| = |g0n)||x − xn−1| ≤ k|x − xn−1|, where ξn∈ (a, b). It follows that

|xn− x| ≤ k|xn−1− x| ≤ k2|xn−2− x| ≤ · · · ≤ kn|x0− x|. (1) Since 0 < k < 1, we have

n→∞lim kn= 0 and

n→∞lim |xn− x| ≤ lim

n→∞kn|x0− x| = 0.

Hence, {xn}n=0converges to x.

(24)

Corollary 9

If g satisfies the hypotheses of above theorem, then

|x − xn| ≤ knmax{x0− a, b − x0} and

|xn− x| ≤ kn

1 − k|x1− x0|, ∀ n ≥ 1.

Proof: From (1),

|xn− x| ≤ kn|x0− x| ≤ knmax{x0− a, b − x0}.

For n ≥ 1, using the Mean Values Theorem,

|xn+1− xn| = |g(xn) − g(xn−1)| ≤ k|xn− xn−1| ≤ · · · ≤ kn|x1− x0|.

(25)

Thus, for m > n ≥ 1,

|xm− xn| = |xm− xm−1+ xm−1− · · · + xn+1− xn|

≤ |xm− xm−1| + |xm−1− xm−2| + · · · + |xn+1− xn|

≤ km−1|x1− x0| + km−2|x1− x0| + · · · + kn|x1− x0|

= kn|x1− x0| 1 + k + k2+ · · · + km−n−1 . It implies that

|x − xn| = lim

m→∞|xm− xn| ≤ lim

m→∞kn|x1− x0|

m−n−1

X

j=0

kj

≤ kn|x1− x0|

X

j=0

kj = kn

1 − k|x1− x0|.

(26)

Example 10

For previous example,

f (x) = x3+ 4x2− 10 = 0.

For g1(x) = x − x3− 4x2+ 10, we have g1(1) = 6 and g1(2) = −12, so g1([1, 2]) * [1, 2]. Moreover,

g10(x) = 1 − 3x2− 8x ⇒ |g10(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

(27)

For g3(x) = 12(10 − x3)1/2, ∀ x ∈ [1, 1.5], g30(x) = −3

4x2(10 − x3)−1/2< 0, ∀ x ∈ [1, 1.5], so g3 is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g3(1.5) ≤ g3(x) ≤ g3(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g03(x)| ≤ |g03(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

(28)

For g4(x) =p10/(4 + x), we have r10

6 ≤ g4(x) ≤ r10

5 , ∀ x ∈ [1, 2] ⇒ g4([1, 2]) ⊆ [1, 2]

Moreover,

|g04(x)| =

√ −5

10(4 + x)3/2

≤ 5

√10(5)3/2 < 0.15, ∀ x ∈ [1, 2].

The bound of |g04(x)|is much smaller than the bound of |g30(x)|, which explains the more rapid convergence using g4.

(29)

Suppose that f : R → R and f ∈ C2[a, b], i.e., f00exists and is continuous. If f (x) = 0and x= x + hwhere h is small, then byTaylor’stheorem

0 = f (x) = f (x + h)

= f (x) + f0(x)h + 1

2f00(x)h2+ 1

3!f000(x)h3+ · · ·

= f (x) + f0(x)h + O(h2).

Sincehissmall, O(h2)is negligible. It is reasonable to drop O(h2)terms. This implies

f (x) + f0(x)h ≈ 0 and h ≈ −f (x)

f0(x), if f0(x) 6= 0.

Hence

x + h = x − f (x) f0(x) is a better approximation to x.

(30)

This sets the stage for theNewton-Rapbson’smethod, which starts with an initial approximation x0 and generates the sequence {xn}n=0defined by

xn+1 = xn− f (xn) f0(xn).

Since the Taylor’s expansion of f (x) at xkis given by f (x) = f (xk) + f0(xk)(x − xk) +1

2f00(xk)(x − xk)2+ · · · . At xk, one uses thetangent line

y = `(x) = f (xk) + f0(xk)(x − xk)

toapproximate the curveof f (x) and uses the zero of the tangent line to approximate the zero of f (x).

(31)

Newton’s Method

Given x0, tolerance T OL, maximum number of iteration M . Set i = 1 and x = x0− f (x0)/f0(x0).

While i ≤ M and |x − x0| ≥ T OL

Set i = i + 1, x0 = xand x = x0− f (x0)/f0(x0).

End While

(32)

Three stopping-technique inequalities

(a). |xn− xn−1| < ε, (b). |xn− xn−1|

|xn| < ε, xn6= 0, (c). |f (xn)| < ε.

Note that Newton’s method for solving f (x) = 0 xn+1= xn− f (xn)

f0(xn), for n ≥ 1 is just a special case of functional iteration in which

g(x) = x − f (x) f0(x).

(33)

Example 11

The following table shows the convergence behavior of

Newton’s method applied to solving f (x) = x2− 1 = 0. Observe the quadratic convergence rate.

n xn |en| ≡ |1 − xn|

0 2.0 1

1 1.25 0.25

2 1.025 2.5e-2

3 1.0003048780488 3.048780488e-4 4 1.0000000464611 4.64611e-8

5 1.0 0

(34)

Theorem 12

Assumef (x) = 0, f0(x) 6= 0andf (x),f0(x)andf00(x)are continuouson Nε(x). Then if x0is chosensufficiently closeto x, then



xn+1 = xn− f (xn) f0(xn)



→ x.

Proof: Define

g(x) = x − f (x) f0(x). Find an interval [x− δ, x+ δ]such that

g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ]

and

|g0(x)| ≤ k < 1, ∀ x ∈ (x− δ, x+ δ).

(35)

Since f0 is continuous and f0(x) 6= 0, it implies that ∃ δ1 > 0 such that f0(x) 6= 0 ∀ x ∈ [x− δ1, x+ δ1] ⊆ [a, b]. Thus, g is defined and continuous on [x− δ1, x+ δ1]. Also

g0(x) = 1 − f0(x)f0(x) − f (x)f00(x)

[f0(x)]2 = f (x)f00(x) [f0(x)]2 , for x ∈ [x− δ1, x+ δ1]. Since f00 is continuous on [a, b], we have g0 is continuous on [x− δ1, x+ δ1].

By assumption f (x) = 0, so

g0(x) = f (x)f00(x)

|f0(x)|2 = 0.

Since g0 is continuous on [x− δ1, x+ δ1]and g0(x) = 0, ∃ δ with 0 < δ < δ1 and k ∈ (0, 1) such that

|g0(x)| ≤ k, ∀ x ∈ [x− δ, x+ δ].

(36)

Claim: g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ].

If x ∈ [x− δ, x+ δ], then, by the Mean Value Theorem, ∃ ξ between x and x such that

|g(x) − g(x)| = |g0(ξ)||x − x|.

It implies that

|g(x) − x| = |g(x) − g(x)| = |g0(ξ)||x − x|

≤ k|x − x| < |x − x| < δ.

Hence, g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ].

By the Fixed-Point Theorem, the sequence {xn}n=0defined by xn= g(xn−1) = xn−1− f (xn−1)

f0(xn−1), for n ≥ 1, converges to x for any x0 ∈ [x− δ, x+ δ].

(37)

Example 13

When Newton’s method applied to f (x) = cos x with starting point x0 = 3, which is close to the root π2 of f , it produces x1 = −4.01525, x2= −4.8526, · · · ,which converges to another root −2 .

5 4 3 2 1 0 1 2 3 4 5

1. 5 0 1.5

x0 y = c os ( x)

x*

(38)

### Secant method

In many applications, the derivative f0(x)is very expensive to compute, or the function f (x) is not given in an algebraic formula so that f0(x)is not available.

By definition,

f0(xn−1) = lim

x→xn−1

f (x) − f (xn−1) x − xn−1 . Letting x = xn−2, we have

f0(xn−1) ≈ f (xn−2) − f (xn−1)

xn−2− xn−1 = f (xn−1) − f (xn−2) xn−1− xn−2 . Using this approximation for f0(xn−1)in Newton’s formula gives

xn= xn−1−f (xn−1)(xn−1− xn−2) f (xn−1) − f (xn−2) ,

38 / 68

(39)

From geometric point of view, we use asecant linethrough xn−1andxn−2 instead of the tangent line to approximate the function at the point xn−1.

The slope of the secant line is

sn−1 = f (xn−1) − f (xn−2) xn−1− xn−2 and the equation is

M (x) = f (xn−1) + sn−1(x − xn−1).

The zero of the secant line x = xn−1−f (xn−1)

sn−1 = xn−1− f (xn−1) xn−1− xn−2 f (xn−1) − f (xn−2) is then used as a new approximate xn.

(40)

Secant Method

Given x0, x1, tolerance T OL, maximum number of iteration M . Set i = 2; y0 = f (x0); y1= f (x1);

x = x1− y1(x1− x0)/(y1− y0).

While i ≤ M and |x − x1| ≥ T OL

Set i = i + 1; x0 = x1; y0 = y1; x1= x; y1 = f (x);

x = x1− y1(x1− x0)/(y1− y0).

End While

(41)

Method of False Position

1 Choose initial approximations x0 and x1 with f (x0)f (x1) < 0.

2 x2= x1− f (x1)(x1− x0)/(f (x1) − f (x0))

3 Decide which secant line to use to compute x3: If f (x2)f (x1) < 0, then x1and x2 bracket a root, i.e.,

x3= x2− f (x2)(x2− x1)/(f (x2) − f (x1)) Else, x0 and x2 bracket a root, i.e.,

x3= x2− f (x2)(x2− x0)/(f (x2) − f (x0)) End if

(42)

Method of False Position

Given x0, x1, tolerance T OL, maximum number of iteration M . Set i = 2; y0 = f (x0); y1= f (x1); x = x1− y1(x1− x0)/(y1− y0).

While i ≤ M and |x − x1| ≥ T OL Set i = i + 1; y = f (x).

If y · y1< 0, then set x0= x1; y0 = y1.

Set x1 = x; y1 = y; x = x1− y1(x1− x0)/(y1− y0).

End While

(43)

### Error analysis for iterative methods

Definition 14

Let{xn} → x. If there are positive constantscandαsuch that

n→∞lim

|xn+1− x|

|xn− x|α = c,

then we say therate of convergenceis oforderα.

We say that the rate of convergence is

1 linearifα = 1and 0 < c < 1.

2 superlinearif

n→∞lim

|xn+1− x|

|xn− x| = 0;

3 quadraticifα = 2. 43 / 68

(44)

Suppose that {xn}n=0and {˜xn}n=0are linearly and quadratically convergent to x, respectively, with the same constant c = 0.5. For simplicity, suppose that

|xn+1− x|

|xn− x| ≈ c and |˜xn+1− x|

|˜xn− x|2 ≈ c.

These imply that

|xn− x| ≈ c|xn−1− x| ≈ c2|xn−2− x| ≈ · · · ≈ cn|x0− x|, and

|˜xn− x| ≈ c|˜xn−1− x|2 ≈ cc|˜xn−2− x|22

= c3|˜xn−2− x|4

≈ c3c|˜xn−3− x|24

= c7|˜xn−3− x|8

≈ · · · ≈ c2n−1|˜x0− x|2n.

(45)

Remark

Quadratically convergent sequences generally converge much more quickly thank those that converge only linearly.

Theorem 15

Let g ∈ C[a, b] with g([a, b]) ⊆ [a, b]. Suppose that g0 is continuous on (a, b) and ∃ k ∈ (0, 1) such that

|g0(x)| ≤ k, ∀ x ∈ (a, b).

Ifg0(x) 6= 0, then for any x0∈ [a, b], the sequence xn= g(xn−1), for n ≥ 1

converges onlylinearlyto the unique fixed point x in [a, b].

(46)

Proof:

By the Fixed-Point Theorem, the sequence {xn}n=0 converges to x.

Since g0 exists on (a, b), by the Mean Value Theorem, ∃ ξn between xnand x such that

xn+1− x = g(xn) − g(x) = g0n)(xn− x).

∵ {xn}n=0→ x ⇒ {ξn}n=0→ x Since g0 is continuous on (a, b), we have

n→∞lim g0n) = g0(x).

Thus,

n→∞lim

|xn+1− x|

|xn− x| = lim

n→∞|g0n)| = |g0(x)|.

Hence, if g0(x) 6= 0, fixed-point iteration exhibits linear convergence.

(47)

Theorem 16

Let x be a fixed point of g and I be an open interval with x ∈ I. Suppose thatg0(x) = 0and g00is continuous with

|g00(x)| < M, ∀ x ∈ I.

Then ∃ δ > 0 such that

{xn= g(xn−1)}n=1 → x for x0 ∈ [x− δ, x+ δ]

|xn+1− x| < M

2 |xn− x|2, for sufficiently large n.

(48)

Proof:

Since g0(x) = 0and g0 is continuous on I, ∃ δ such that [x− δ, x+ δ] ⊂ Iand

|g0(x)| ≤ k < 1, ∀ x ∈ [x− δ, x+ δ].

In the proof of the convergence for Newton’s method, we have

{xn}n=0⊂ [x− δ, x+ δ].

Consider the Taylor expansion of g(xn)at x xn+1= g(xn) = g(x) + g0(x)(xn− x) +g00(ξ)

2 (xn− x)2

= x+g00(ξ)

2 (xn− x)2, where ξ lies between xnand x.

(49)

Since

|g0(x)| ≤ k < 1, ∀ x ∈ [x− δ, x+ δ]

and

g([x− δ, x+ δ]) ⊆ [x− δ, x+ δ], it follows that {xn}n=0converges to x.

But ξnis between xnand xfor each n, so {ξn}n=0also converges to x and

n→∞lim

|xn+1− x|

|xn− x|2 = |g00(x)|

2 < M 2 .

It implies that {xn}n=0 is quadratically convergent to xif g00(x) 6= 0and

|xn+1− x| < M

2 |xn− x|2, for sufficiently large n.

(50)

For Newton’s method, g(x) = x − f (x)

f0(x) ⇒ g0(x) = 1 − f0(x)

f0(x)+f (x)f00(x)

(f0(x))2 = f (x)f00(x) (f0(x))2 It follows that g0(x) = 0. Hence Newton’s method is locally quadratically convergent.

(51)

### Error Analysis of Secant Method

Reference: D. Kincaid and W. Cheney, ”Numerical analysis”

Let x denote the exact solution of f (x) = 0, ek= xk− x be the error at the k-th step. Then

ek+1 = xk+1− x

= xk− f (xk) xk− xk−1

f (xk) − f (xk−1) − x

= 1

f (xk) − f (xk−1)[(xk−1− x)f (xk) − (xk− x)f (xk−1)]

= 1

f (xk) − f (xk−1)(ek−1f (xk) − ekf (xk−1))

= ekek−1

1

ekf (xk) − e1

k−1f (xk−1)

xk− xk−1 · xk− xk−1 f (xk) − f (xk−1)

!

(52)

To estimate the numerator

1

ekf (xk)− 1

ek−1f (xk−1)

xk−xk−1 , we apply Taylor’s Theorem

f (xk) = f (x+ ek) = f (x) + f0(x)ek+1

2f00(x)e2k+ O(e3k), to get

1 ek

f (xk) = f0(x) +1

2f00(x)ek+ O(e2k).

Similarly, 1

ek−1f (xk−1) = f0(x) +1

2f00(x)ek−1+ O(e2k−1).

Hence 1

ekf (xk) − 1

ek−1f (xk−1) ≈ 1

2(ek− ek−1)f00(x).

Since xk− xk−1 = ek− ek−1and xk− xk−1

f (x ) − f (x ) → 1 f0(x),

(53)

we have

ek+1 ≈ ekek−1

1

2(ek− ek−1)f00(x) ek− ek−1 · 1

f0(x)

!

= 1 2

f00(x) f0(x)ekek−1

≡ Cekek−1. (2)

To estimate the convergence rate, we assume

|ek+1| ≈ η|ek|α, where η > 0 and α > 0 are constants, i.e.,

|ek+1|

η|ek|α → 1 as k → ∞.

Then |ek| ≈ η|ek−1|αwhich implies |ek−1| ≈ η−1/α|ek|1/α. Hence (2) gives

η|ek|α≈ C|ek−1/α|ek|1/α =⇒ C−1η1+α1 ≈ |ek|1−α+α1.

(54)

Since |ek| → 0 as k → ∞, and C−1η1+α1 is a nonzero constant, 1 − α + 1

α = 0 =⇒ α = 1 +√ 5

2 ≈ 1.62.

This result implies that C−1η1+α1 → 1 and η → C1+αα = f00(x)

2f0(x)

0.62

. In summary, we have shown that

|ek+1| = η|ek|α, α ≈ 1.62, that is, therate of convergenceissuperlinear.

Rate of convergence

secantmethod: superlinear Newton’smethod: quadratic bisectionmethod: linear

(55)

Each iteration of method requires

secant method: one function evaluation

Newton’s method: two function evaluation, namely, f (xk) and f0(xk).

⇒ two steps of secant method are comparable to one step of Newton’s method. Thus

|ek+2| ≈ η|ek+1|α ≈ η1+α|ek|3+

5

2 ≈ η1+α|ek|2.62.

⇒ secant method is more efficient than Newton’s method.

Remark

Two steps of secant method would require a little more work than one step of Newton’s method.

(56)

Aitken’s ∆2 method

Accelerate the convergence of a sequence that islinearly convergent.

Suppose {yn}n=0is a linearly convergent sequence with limit y. Construct a sequence {ˆyn}n=0that converges more rapidly to y than {yn}n=0.

For n sufficiently large, yn+1− y

yn− y ≈ yn+2− y yn+1− y. Then

(yn+1− y)2≈ (yn+2− y)(yn− y), so

y2n+1− 2yn+1y + y2 ≈ yn+2yn− (yn+2+ yn)y + y2

(57)

and

(yn+2+ yn− 2yn+1)y ≈ yn+2yn− y2n+1. Solving for y gives

y ≈ yn+2yn− yn+12 yn+2− 2yn+1+ yn

= ynyn+2− 2ynyn+1+ y2n− yn2+ 2ynyn+1− y2n+1 yn+2− 2yn+1+ yn

= yn(yn+2− 2yn+1+ yn) − (yn+1− yn)2 (yn+2− yn+1) − (yn+1− yn)

= yn− (yn+1− yn)2

(yn+2− yn+1) − (yn+1− yn). Aitken’s ∆2 method

ˆ

yn= yn− (yn+1− yn)2

(yn+2− yn+1) − (yn+1− yn). (3)

(58)

Example 17

The sequence {yn= cos(1/n)}n=1converges linearly to y = 1.

n ynn

1 0.54030 0.96178 2 0.87758 0.98213 3 0.94496 0.98979 4 0.96891 0.99342 5 0.98007 0.99541 6 0.98614

7 0.98981

{ˆyn}n=1converges more rapidly to y = 1 than {yn}n=1.

(59)

Definition 18

For a given sequence {yn}n=0, the forward difference ∆ynis defined by

∆yn= yn+1− yn, for n ≥ 0.

Higher powers of ∆ are defined recursively by

kyn= ∆(∆k−1yn), for k ≥ 2.

The definition implies that

2yn= ∆(yn+1− yn) = ∆yn+1− ∆yn= (yn+2− yn+1) − (yn+1− yn).

So the formula for ˆynin (3) can be written as ˆ

yn= yn−(∆yn)2

2yn

, for n ≥ 0.

(60)

Theorem 19

Suppose {yn}n=0 → ylinearlyand

n→∞lim

yn+1− y yn− y < 1.

Then {ˆyn}n=0→ y faster than {yn}n=0in the sense that

n→∞lim ˆ yn− y yn− y = 0.

Aitken’s ∆2method constructs the terms in order:

y0, y1 = g(y0), y2= g(y1), yˆ0= {∆2}(y0), y3 = g(y2), ˆ

y1 = {∆2}(y1), . . . .

⇒ Assume |ˆy0− y| < |y2− y|

(61)

Steffensen’s method constructs the terms in order:

y0(0)≡ y0, y1(0) = g(y(0)0 ), y2(0)= g(y1(0)), y0(1)= {∆2}(y0(0)), y(1)1 = g(y(1)0 ), y2(1)= g(y(1)1 ), . . . . Steffensen’s method (To find a solution of y = g(y))

Given y0, tolerance T ol, max. number of iteration M . Set i = 1.

While i ≤ M

Set y1= g(y0); y2= g(y1); y = y0− (y1− y0)2/(y2− 2y1+ y0).

If |y − y0| < T ol, then STOP.

Set i = i + 1; y0 = y.

End While Theorem 20

Suppose x = g(x) has solution x with g0(x) 6= 1. If ∃ δ > 0 such that g ∈ C3[x− δ, x+ δ], then Steffensen’s method gives quadraticconvergence for any x0 ∈ [x− δ, x+ δ].

(62)

### Zeros of polynomials and M ¨uller’s method

• Horner’s method:

Let

P (x) = a0+ a1x + a2x2+ · · · + an−1xn−1+ anxn

= a0+ x (a1+ x (a2+ · · · + x (an−1+ anx) · · · )) . If

bn = an,

bk = ak+ bk+1x0, for k = n − 1, n − 2, . . . , 1, 0, then

b0= a0+ b1x0= a0+ (a1+ b2x0) x0 = · · · = P (x0).

Consider

Q(x) = b1+ b2x + · · · + bnxn−1.

(63)

Then

b0+ (x − x0)Q(x) = b0+ (x − x0) b1+ b2x + · · · + bnxn−1

= (b0− b1x0) + (b1− b2x0)x + · · · + (bn−1− bnx0)xn−1+ bnxn

= a0+ a1x + · · · + anxn= P (x).

Differentiating P (x) with respect to x gives

P0(x) = Q(x) + (x − x0)Q0(x) and P0(x0) = Q(x0).

Use Newton-Raphson method to find an approximate zero of P (x):

xk+1= xk−P (xk)

Q(xk), ∀ k = 0, 1, 2, . . . . Similarly, let

cn = bn= an,

ck = bk+ ck+1xk, for k = n − 1, n − 2, . . . , 1, then c1 = Q(xk).

(64)

Horner’s method(Evaluate y = P (x0)and z = P0(x0)) Set y = an; z = an.

For j = n − 1, n − 2, . . . , 1 Set y = aj+ yx0; z = y + zx0. End for

Set y = a0+ yx0. If xN is an approximate zero of P , then

P (x) = (x − xN)Q(x) + b0 = (x − xN)Q(x) + P (xN)

≈ (x − xN)Q(x) ≡ (x − ˆx1)Q1(x).

So x − ˆx1is an approximate factor of P (x) and we can find a second approximate zero of P by applying Newton’s method to Q1(x). The procedure is called deflation.

(65)

• M ¨uller’s method for complex root:

Theorem 21

If z = a + ib is a complex zero of multiplicity m of P (x) with real coefficients, then ¯z = a − biis also a zero of multiplicity m of P (x)and (x2− 2ax + a2+ b2)m is a factor of P (x).

Secant method: Given p0 and p1, determine p2 as the intersection of the x-axis with the line through (p0, f (p0))and (p1, f (p1)).

M ¨uller’s method: Given p0, p1

and p2, determine p3 by the intersection of the x-axis with the parabola through (p0, f (p0)), (p1, f (p1))and (p2, f (p2)).

(66)

Let

P (x) = a(x − p2)2+ b(x − p2) + c

that passes through (p0, f (p0)), (p1, f (p1))and (p2, f (p2)). Then f (p0) = a(p0− p2)2+ b(p0− p2) + c,

f (p1) = a(p1− p2)2+ b(p1− p2) + c, f (p2) = a(p2− p2)2+ b(p2− p2) + c = c.

It implies that c = f (p2),

b = (p0− p2)2[f (p1) − f (p2)] − (p1− p2)2[f (p0) − f (p2)]

(p0− p2)(p1− p2)(p0− p1) , a = (p1− p2) [f (p0) − f (p2)] − (p0− p2) [f (p1) − f (p2)]

(p0− p2)(p1− p2)(p0− p1) .

(67)

To determine p3, a zero of P , we apply the quadratic formula to P (x) = 0and get

p3− p2 = 2c b ±√

b2− 4ac. Choose

p3 = p2+ 2c b + sgn(b)√

b2− 4ac

such that the denominator will be largest and result in p3 selected as the closest zero of P to p2.

(68)

M ¨uller’s method (Find a solution of f (x) = 0)

Given p0, p1, p2; tolerance T OL; maximum number of iterations M Set h1 = p1− p0; h2 = p2− p1;

δ1= (f (p1) − f (p0))/h1; δ2 = (f (p2) − f (p1))/h2; d = (δ2− δ1)/(h2+ h1); i = 3.

While i ≤ M

Set b = δ2+ h2d; D =pb2− 4f (p2)d.

If |b − D| < |b + D|, then set E = b + D else set E = b − D.

Set h = −2f (p2)/E; p = p2+ h.

If |h| < T OL, then STOP.

Set p0= p1; p1 = p2; p2 = p; h1= p1− p0; h2= p2− p1; δ1= (f (p1) − f (p0))/h1; δ2 = (f (p2) − f (p1))/h2; d = (δ2− δ1)/(h2+ h1); i = i + 1.

End while

Updating...

## References

Related subjects :