師大

Tsung-Ming Huang

Department of Mathematics National Taiwan Normal University, Taiwan

August 28, 2011

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Bisection Method**

**Idea**

Iff (x) ∈ C[a, b]andf (a)f (b) < 0, then∃ c ∈ (a, b)such that f (c) = 0.

師大

**Bisection method algorithm**

Given f (x) defined on (a, b), the maximal number of iterations M, and stop criteria δ and ε, this algorithm tries to locate one root of f (x).

Compute u = f (a), v = f (b), and e = b − a
**If sign(u) = sign(v), then stop**

**For k = 1, 2, . . . , M**

e = e/2, c = a + e, w = f (c)
**If |e| < δ or |w| < ε, then stop**
**If sign(w) 6= sign(u)**

b = c, v = w
**Else**

a = c, u = w
**End If**

**End For**

師大

Let {c_{n}} be the sequence of numbers produced. The algorithm
should stop if one of the following conditions is satisfied.

**1** the iteration number k > M ,

**2** |c_{k}− c_{k−1}| < δ, or

**3** |f (c_{k})| < ε.

Let [a_{0}, b_{0}], [a_{1}, b_{1}], . . .denote the successive intervals produced
by the bisection algorithm. Then

a = a_{0}≤ a_{1} ≤ a_{2} ≤ · · · ≤ b_{0}= b

⇒ {a_{n}} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

師大

Since

b_{1}− a_{1} = 1

2(b_{0}− a_{0})
b_{2}− a_{2} = 1

2(b_{1}− a_{1}) = 1

4(b_{0}− a_{0})
...

b_{n}− a_{n} = 1

2^{n}(b_{0}− a_{0})
hence

n→∞lim bn− lim

n→∞an= lim

n→∞(bn− a_{n}) = lim

n→∞

1

2^{n}(b0− a_{0}) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

師大

On the other hand,

f (a_{n})f (b_{n}) ≤ 0

⇒ lim

n→∞f (a_{n})f (b_{n}) = f^{2}(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {a_{n}} and {b_{n}} is a zero of
f in [a, b]. Let c_{n}= ^{1}_{2}(a_{n}+ b_{n}). Then

|z − c_{n}| =
lim

n→∞a_{n}−1

2(a_{n}+ b_{n})

= 1 2

h

n→∞lim a_{n}− b_{n}i
+ 1

2 h

n→∞lim a_{n}− a_{n}i

≤ max lim

n→∞an− b_{n}
,

lim

n→∞an− a_{n}

≤ |b_{n}− a_{n}| = 1

2^{n}|b_{0}− a_{0}|.

This proves the following theorem.

師大

**Theorem 1**

Let{[a_{n}, bn]}denote the intervals produced by the bisection
algorithm. Then lim

n→∞anand lim

n→∞bnexist, areequal, and represent azerooff (x). If

z = lim

n→∞an= lim

n→∞bn and cn= 1

2(an+ bn), then

|z − c_{n}| ≤ 1

2^{n}(b_{0}− a_{0}) .
**Remark**

{c_{n}}convergestozwith therateofO(2^{−n}).

師大

**Example 2**

How many steps should be taken to compute a root of
f (x) = x^{3}+ 4x^{2}− 10 = 0 on [1, 2] with relative error 10^{−3}?
solution: Seek an n such that

|z − c_{n}|

|z| ≤ 10^{−3} ⇒ |z − c_{n}| ≤ |z| × 10^{−3}.
Since z ∈ [1, 2], it is sufficient to show

|z − c_{n}| ≤ 10^{−3}.
That is, we solve

2^{−n}(2 − 1) ≤ 10^{−3} ⇒ −n log_{10}2 ≤ −3
which gives n ≥ 10.

師大

**Fixed-Point Iteration**

**Definition 3**

xis called afixed pointof a given functionf iff (x) = x.

**Root-finding problems and fixed-point problems**
Find x^{∗} such that f (x^{∗}) = 0.

Let g(x) = x − f (x). Then g(x^{∗}) = x^{∗}− f (x^{∗}) = x^{∗}.

⇒ x^{∗}is a fixed point for g(x).

Find x^{∗} such that g(x^{∗}) = x^{∗}.
Define f (x) = x − g(x) so that
f (x^{∗}) = x^{∗}− g(x^{∗}) = x^{∗}− x^{∗}= 0

⇒ x^{∗}is a zero of f (x).

師大

**Example 4**

The function g(x) = x^{2}− 2, for −2 ≤ x ≤ 3, has fixed points at
x = −1and x = 2 since

g(−1) = (−1)^{2}− 2 = −1 and g(2) = 2^{2}− 2 = 2.

師大

**Theorem 5 (Existence and uniqueness)**

**1** If g ∈ C[a, b] such thata ≤ g(x) ≤ bfor all x ∈ [a, b], then g
hasa fixed point in [a, b].

**2** If, in addition, g^{0}(x)exists in (a, b) and there exists a
positive constantM < 1such that|g^{0}(x)| ≤ M < 1for all
x ∈ (a, b). Then the fixed point isunique.

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0. That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b]. By the Mean-Value theorem, there exists ξ between p and q such that

g^{0}(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g^{0}(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is
unique.

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

Let p be such unique fixed point of g. Then
p = g(p) = p^{2}− 1

3 ⇒ p^{2}− 3p − 1 = 0

⇒ p = 1 2(3 −√

13).

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x. However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration**

Given x_{0}, tolerance T OL, maximum number of iteration M .
Set i = 1 and x = g(x_{0}).

While i ≤ M and |x − x_{0}| ≥ T OL
Set i = i + 1, x_{0} = xand x = g(x_{0}).

End While

師大

**Example 7**
The equation

x^{3}+ 4x^{2}− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g_{1}(x) ≡ x − f (x) = x − x^{3}− 4x^{2}+ 10

(b) x = g_{2}(x) = ^{10}_{x} − 4x1/2

x^{3} = 10 − 4x^{2} ⇒ x^{2} = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

師大

(c) x = g_{3}(x) = ^{1}_{2} 10 − x^{3}1/2

4x^{2} = 10 − x^{3} ⇒ x = ±1

2 10 − x^{3}1/2

(d) x = g_{4}(x) =

10 4+x

1/2

x^{2}(x + 4) = 10 ⇒ x = ±

10 4 + x

1/2

(e) x = g_{5}(x) = x −^{x}^{3}_{3x}^{+4x}2+8x^{2}^{−10}

x = g_{5}(x) ≡ x − f (x)
f^{0}(x)

師大

Results of the fixed-point iteration with initial point x_{0} = 1.5

師大

**Theorem 8 (Fixed-point Theorem)**

Let g ∈ [a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b]. Suppose
that g^{0}exists on (a, b) and that ∃ k with0 < k < 1such that

|g^{0}(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x_{0} in [a, b],

xn= g(xn−1), n ≥ 1, converges to the unique fixed point x in [a, b].

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {x_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b). It follows that

|x_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞}_{n=0}converges to x.

師大

**Corollary 9**

If g satisfies the hypotheses of above theorem, then

|x − x_{n}| ≤ k^{n}max{x_{0}− a, b − x_{0}}
and

|x_{n}− x| ≤ k^{n}

1 − k|x_{1}− x_{0}|, ∀ n ≥ 1.

Proof: From (1),

|x_{n}− x| ≤ k^{n}|x_{0}− x| ≤ k^{n}max{x0− a, b − x_{0}}.

For n ≥ 1, using the Mean Values Theorem,

|x_{n+1}− x_{n}| = |g(x_{n}) − g(x_{n−1})| ≤ k|x_{n}− x_{n−1}| ≤ · · · ≤ k^{n}|x_{1}− x_{0}|.

師大

Thus, for m > n ≥ 1,

|x_{m}− x_{n}| = |x_{m}− x_{m−1}+ xm−1− · · · + x_{n+1}− x_{n}|

≤ |x_{m}− x_{m−1}| + |x_{m−1}− x_{m−2}| + · · · + |x_{n+1}− x_{n}|

≤ k^{m−1}|x_{1}− x_{0}| + k^{m−2}|x_{1}− x_{0}| + · · · + k^{n}|x_{1}− x_{0}|

= k^{n}|x_{1}− x_{0}| 1 + k + k^{2}+ · · · + k^{m−n−1} .
It implies that

|x − x_{n}| = lim

m→∞|x_{m}− x_{n}| ≤ lim

m→∞k^{n}|x_{1}− x_{0}|

m−n−1

X

j=0

k^{j}

≤ k^{n}|x_{1}− x_{0}|

∞

X

j=0

k^{j} = k^{n}

1 − k|x_{1}− x_{0}|.

師大

**Example 10**

For previous example,

f (x) = x^{3}+ 4x^{2}− 10 = 0.

For g_{1}(x) = x − x^{3}− 4x^{2}+ 10, we have
g_{1}(1) = 6 and g_{1}(2) = −12,
so g_{1}([1, 2]) * [1, 2]. Moreover,

g_{1}^{0}(x) = 1 − 3x^{2}− 8x ⇒ |g_{1}^{0}(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{4}(x) =p10/(4 + x), we have
r10

6 ≤ g_{4}(x) ≤
r10

5 , ∀ x ∈ [1, 2] ⇒ g_{4}([1, 2]) ⊆ [1, 2]

Moreover,

|g^{0}_{4}(x)| =

√ −5

10(4 + x)^{3/2}

≤ 5

√10(5)^{3/2} < 0.15, ∀ x ∈ [1, 2].

The bound of |g^{0}_{4}(x)|is much smaller than the bound of |g_{3}^{0}(x)|,
which explains the more rapid convergence using g_{4}.

師大

Suppose that f : R → R and f ∈ C^{2}[a, b], i.e., f^{00}exists and is
continuous. If f (x^{∗}) = 0and x^{∗}= x + hwhere h is small, then
byTaylor’stheorem

0 = f (x^{∗}) = f (x + h)

= f (x) + f^{0}(x)h + 1

2f^{00}(x)h^{2}+ 1

3!f^{000}(x)h^{3}+ · · ·

= f (x) + f^{0}(x)h + O(h^{2}).

Sincehissmall, O(h^{2})is negligible. It is reasonable to drop
O(h^{2})terms. This implies

f (x) + f^{0}(x)h ≈ 0 and h ≈ −f (x)

f^{0}(x), if f^{0}(x) 6= 0.

Hence

x + h = x − f (x)
f^{0}(x)
is a better approximation to x^{∗}.

師大

This sets the stage for theNewton-Rapbson’smethod, which
starts with an initial approximation x_{0} and generates the
sequence {x_{n}}^{∞}_{n=0}defined by

xn+1 = xn− f (xn)
f^{0}(xn).

Since the Taylor’s expansion of f (x) at x_{k}is given by
f (x) = f (x_{k}) + f^{0}(x_{k})(x − x_{k}) +1

2f^{00}(x_{k})(x − x_{k})^{2}+ · · · .
At x_{k}, one uses thetangent line

y = `(x) = f (xk) + f^{0}(xk)(x − xk)

toapproximate the curveof f (x) and uses the zero of the tangent line to approximate the zero of f (x).

師大

**Newton’s Method**

Given x_{0}, tolerance T OL, maximum number of iteration M .
Set i = 1 and x = x0− f (x_{0})/f^{0}(x0).

While i ≤ M and |x − x_{0}| ≥ T OL

Set i = i + 1, x_{0} = xand x = x_{0}− f (x_{0})/f^{0}(x0).

End While

師大

**Three stopping-technique inequalities**

(a). |x_{n}− x_{n−1}| < ε,
(b). |x_{n}− x_{n−1}|

|x_{n}| < ε, xn6= 0,
(c). |f (x_{n})| < ε.

Note that Newton’s method for solving f (x) = 0
xn+1= xn− f (x_{n})

f^{0}(xn), for n ≥ 1
is just a special case of functional iteration in which

g(x) = x − f (x)
f^{0}(x).

師大

**Example 11**

The following table shows the convergence behavior of

Newton’s method applied to solving f (x) = x^{2}− 1 = 0. Observe
the quadratic convergence rate.

n xn |e_{n}| ≡ |1 − x_{n}|

0 2.0 1

1 1.25 0.25

2 1.025 2.5e-2

3 1.0003048780488 3.048780488e-4 4 1.0000000464611 4.64611e-8

5 1.0 0

師大

**Theorem 12**

Assumef (x^{∗}) = 0, f^{0}(x^{∗}) 6= 0andf (x),f^{0}(x)andf^{00}(x)are
continuouson N_{ε}(x^{∗}). Then if x_{0}is chosensufficiently closeto
x^{∗}, then

xn+1 = xn− f (xn)
f^{0}(x_{n})

→ x^{∗}.

Proof: Define

g(x) = x − f (x)
f^{0}(x).
Find an interval [x^{∗}− δ, x^{∗}+ δ]such that

g([x^{∗}− δ, x^{∗}+ δ]) ⊆ [x^{∗}− δ, x^{∗}+ δ]

and

|g^{0}(x)| ≤ k < 1, ∀ x ∈ (x^{∗}− δ, x^{∗}+ δ).

師大

Since f^{0} is continuous and f^{0}(x^{∗}) 6= 0, it implies that ∃ δ_{1} > 0
such that f^{0}(x) 6= 0 ∀ x ∈ [x^{∗}− δ_{1}, x^{∗}+ δ_{1}] ⊆ [a, b]. Thus, g is
defined and continuous on [x^{∗}− δ_{1}, x^{∗}+ δ1]. Also

g^{0}(x) = 1 − f^{0}(x)f^{0}(x) − f (x)f^{00}(x)

[f^{0}(x)]^{2} = f (x)f^{00}(x)
[f^{0}(x)]^{2} ,
for x ∈ [x^{∗}− δ_{1}, x^{∗}+ δ_{1}]. Since f^{00} is continuous on [a, b], we
have g^{0} is continuous on [x^{∗}− δ_{1}, x^{∗}+ δ1].

By assumption f (x^{∗}) = 0, so

g^{0}(x^{∗}) = f (x^{∗})f^{00}(x^{∗})

|f^{0}(x^{∗})|^{2} = 0.

Since g^{0} is continuous on [x^{∗}− δ_{1}, x^{∗}+ δ_{1}]and g^{0}(x^{∗}) = 0, ∃ δ
with 0 < δ < δ_{1} and k ∈ (0, 1) such that

|g^{0}(x)| ≤ k, ∀ x ∈ [x^{∗}− δ, x^{∗}+ δ].

師大

Claim: g([x^{∗}− δ, x^{∗}+ δ]) ⊆ [x^{∗}− δ, x^{∗}+ δ].

If x ∈ [x^{∗}− δ, x^{∗}+ δ], then, by the Mean Value Theorem, ∃ ξ
between x and x^{∗} such that

|g(x) − g(x^{∗})| = |g^{0}(ξ)||x − x^{∗}|.

It implies that

|g(x) − x^{∗}| = |g(x) − g(x^{∗})| = |g^{0}(ξ)||x − x^{∗}|

≤ k|x − x^{∗}| < |x − x^{∗}| < δ.

Hence, g([x^{∗}− δ, x^{∗}+ δ]) ⊆ [x^{∗}− δ, x^{∗}+ δ].

By the Fixed-Point Theorem, the sequence {x_{n}}^{∞}_{n=0}defined by
x_{n}= g(x_{n−1}) = x_{n−1}− f (x_{n−1})

f^{0}(xn−1), for n ≥ 1,
converges to x^{∗} for any x0 ∈ [x^{∗}− δ, x^{∗}+ δ].

師大

**Example 13**

When Newton’s method applied to f (x) = cos x with starting
point x_{0} = 3, which is close to the root ^{π}_{2} of f , it produces
x_{1} = −4.01525, x_{2}= −4.8526, · · · ,which converges to another
root −^{3π}_{2} .

5 4 3 2 1 0 1 2 3 4 5

1. 5 0 1.5

x_{0}
y = c os ( x)

x^{*}

師大

**Secant method**

**Disadvantage of Newton’s method**

In many applications, the derivative f^{0}(x)is very expensive to
compute, or the function f (x) is not given in an algebraic
formula so that f^{0}(x)is not available.

By definition,

f^{0}(x_{n−1}) = lim

x→xn−1

f (x) − f (x_{n−1})
x − x_{n−1} .
Letting x = xn−2, we have

f^{0}(xn−1) ≈ f (xn−2) − f (xn−1)

x_{n−2}− x_{n−1} = f (xn−1) − f (xn−2)
x_{n−1}− x_{n−2} .
Using this approximation for f^{0}(x_{n−1})in Newton’s formula gives

xn= xn−1−f (xn−1)(xn−1− x_{n−2})
f (xn−1) − f (xn−2) ,

**38 / 68**

師大

From geometric point of view, we use asecant linethrough
xn−1andxn−2 instead of the tangent line to approximate the
function at the point x_{n−1}.

The slope of the secant line is

s_{n−1} = f (x_{n−1}) − f (x_{n−2})
xn−1− x_{n−2}
and the equation is

M (x) = f (xn−1) + sn−1(x − xn−1).

The zero of the secant line
x = x_{n−1}−f (xn−1)

s_{n−1} = x_{n−1}− f (x_{n−1}) xn−1− x_{n−2}
f (x_{n−1}) − f (x_{n−2})
is then used as a new approximate x_{n}.

師大

**Secant Method**

Given x_{0}, x_{1}, tolerance T OL, maximum number of iteration M .
Set i = 2; y_{0} = f (x_{0}); y_{1}= f (x_{1});

x = x1− y_{1}(x1− x_{0})/(y1− y_{0}).

While i ≤ M and |x − x_{1}| ≥ T OL

Set i = i + 1; x_{0} = x_{1}; y_{0} = y_{1}; x_{1}= x; y_{1} = f (x);

x = x1− y_{1}(x1− x_{0})/(y1− y_{0}).

End While

師大

Method of False Position

**1** Choose initial approximations x_{0} and x_{1} with
f (x_{0})f (x_{1}) < 0.

**2** x_{2}= x_{1}− f (x_{1})(x_{1}− x_{0})/(f (x_{1}) − f (x_{0}))

**3** Decide which secant line to use to compute x_{3}:
If f (x_{2})f (x_{1}) < 0, then x_{1}and x_{2} bracket a root, i.e.,

x_{3}= x_{2}− f (x_{2})(x_{2}− x_{1})/(f (x_{2}) − f (x_{1}))
Else, x0 and x2 bracket a root, i.e.,

x3= x2− f (x_{2})(x2− x_{0})/(f (x2) − f (x0))
End if

師大

**Method of False Position**

Given x0, x1, tolerance T OL, maximum number of iteration M .
Set i = 2; y_{0} = f (x_{0}); y_{1}= f (x_{1}); x = x_{1}− y_{1}(x_{1}− x_{0})/(y_{1}− y_{0}).

While i ≤ M and |x − x_{1}| ≥ T OL
Set i = i + 1; y = f (x).

If y · y_{1}< 0, then set x_{0}= x_{1}; y_{0} = y_{1}.

Set x_{1} = x; y_{1} = y; x = x_{1}− y_{1}(x_{1}− x_{0})/(y_{1}− y_{0}).

End While

師大

**Error analysis for iterative methods**

**Definition 14**

Let{x_{n}} → x^{∗}. If there are positive constantscandαsuch that

n→∞lim

|x_{n+1}− x^{∗}|

|x_{n}− x^{∗}|^{α} = c,

then we say therate of convergenceis oforderα.

We say that the rate of convergence is

**1** linearifα = 1and 0 < c < 1.

**2** superlinearif

n→∞lim

|x_{n+1}− x^{∗}|

|x_{n}− x^{∗}| = 0;

**3** quadraticifα = 2. ^{43 / 68}

師大

Suppose that {x_{n}}^{∞}_{n=0}and {˜x_{n}}^{∞}_{n=0}are linearly and
quadratically convergent to x^{∗}, respectively, with the same
constant c = 0.5. For simplicity, suppose that

|x_{n+1}− x^{∗}|

|x_{n}− x^{∗}| ≈ c and |˜xn+1− x^{∗}|

|˜x_{n}− x^{∗}|^{2} ≈ c.

These imply that

|x_{n}− x^{∗}| ≈ c|x_{n−1}− x^{∗}| ≈ c^{2}|x_{n−2}− x^{∗}| ≈ · · · ≈ c^{n}|x_{0}− x^{∗}|,
and

|˜x_{n}− x^{∗}| ≈ c|˜x_{n−1}− x^{∗}|^{2} ≈ cc|˜x_{n−2}− x^{∗}|^{2}2

= c^{3}|˜x_{n−2}− x^{∗}|^{4}

≈ c^{3}c|˜x_{n−3}− x^{∗}|^{2}4

= c^{7}|˜x_{n−3}− x^{∗}|^{8}

≈ · · · ≈ c^{2}^{n}^{−1}|˜x0− x^{∗}|^{2}^{n}.

師大

**Remark**

Quadratically convergent sequences generally converge much more quickly thank those that converge only linearly.

**Theorem 15**

Let g ∈ C[a, b] with g([a, b]) ⊆ [a, b]. Suppose that g^{0} is
continuous on (a, b) and ∃ k ∈ (0, 1) such that

|g^{0}(x)| ≤ k, ∀ x ∈ (a, b).

Ifg^{0}(x^{∗}) 6= 0, then for any x0∈ [a, b], the sequence
xn= g(xn−1), for n ≥ 1

converges onlylinearlyto the unique fixed point x^{∗} in [a, b].

師大

Proof:

By the Fixed-Point Theorem, the sequence {x_{n}}^{∞}_{n=0}
converges to x^{∗}.

Since g^{0} exists on (a, b), by the Mean Value Theorem, ∃ ξ_{n}
between xnand x^{∗} such that

xn+1− x^{∗} = g(xn) − g(x^{∗}) = g^{0}(ξn)(xn− x^{∗}).

∵ {xn}^{∞}_{n=0}→ x^{∗} ⇒ {ξ_{n}}^{∞}_{n=0}→ x^{∗}
Since g^{0} is continuous on (a, b), we have

n→∞lim g^{0}(ξ_{n}) = g^{0}(x^{∗}).

Thus,

n→∞lim

|x_{n+1}− x^{∗}|

|x_{n}− x^{∗}| = lim

n→∞|g^{0}(ξ_{n})| = |g^{0}(x^{∗})|.

Hence, if g^{0}(x^{∗}) 6= 0, fixed-point iteration exhibits linear
convergence.

師大

**Theorem 16**

Let x^{∗} be a fixed point of g and I be an open interval with
x^{∗} ∈ I. Suppose thatg^{0}(x^{∗}) = 0and g^{00}is continuous with

|g^{00}(x)| < M, ∀ x ∈ I.

Then ∃ δ > 0 such that

{x_{n}= g(x_{n−1})}^{∞}_{n=1} → x^{∗} for x_{0} ∈ [x^{∗}− δ, x^{∗}+ δ]

at leastquadratically. Moreover,

|x_{n+1}− x^{∗}| < M

2 |x_{n}− x^{∗}|^{2}, for sufficiently large n.

師大

Proof:

Since g^{0}(x^{∗}) = 0and g^{0} is continuous on I, ∃ δ such that
[x^{∗}− δ, x^{∗}+ δ] ⊂ Iand

|g^{0}(x)| ≤ k < 1, ∀ x ∈ [x^{∗}− δ, x^{∗}+ δ].

In the proof of the convergence for Newton’s method, we have

{x_{n}}^{∞}_{n=0}⊂ [x^{∗}− δ, x^{∗}+ δ].

Consider the Taylor expansion of g(x_{n})at x^{∗}
x_{n+1}= g(x_{n}) = g(x^{∗}) + g^{0}(x^{∗})(x_{n}− x^{∗}) +g^{00}(ξ)

2 (x_{n}− x^{∗})^{2}

= x^{∗}+g^{00}(ξ)

2 (x_{n}− x^{∗})^{2},
where ξ lies between xnand x^{∗}.

師大

Since

|g^{0}(x)| ≤ k < 1, ∀ x ∈ [x^{∗}− δ, x^{∗}+ δ]

and

g([x^{∗}− δ, x^{∗}+ δ]) ⊆ [x^{∗}− δ, x^{∗}+ δ],
it follows that {x_{n}}^{∞}_{n=0}converges to x^{∗}.

But ξnis between xnand x^{∗}for each n, so {ξn}^{∞}_{n=0}also
converges to x^{∗} and

n→∞lim

|x_{n+1}− x^{∗}|

|x_{n}− x^{∗}|^{2} = |g^{00}(x^{∗})|

2 < M 2 .

It implies that {xn}^{∞}_{n=0} is quadratically convergent to x^{∗}if
g^{00}(x^{∗}) 6= 0and

|x_{n+1}− x^{∗}| < M

2 |x_{n}− x^{∗}|^{2}, for sufficiently large n.

師大

For Newton’s method, g(x) = x − f (x)

f^{0}(x) ⇒ g^{0}(x) = 1 − f^{0}(x)

f^{0}(x)+f (x)f^{00}(x)

(f^{0}(x))^{2} = f (x)f^{00}(x)
(f^{0}(x))^{2}
It follows that g^{0}(x^{∗}) = 0. Hence Newton’s method is locally
quadratically convergent.

師大

**Error Analysis of Secant Method**

Reference: D. Kincaid and W. Cheney, ”Numerical analysis”

Let x^{∗} denote the exact solution of f (x) = 0, e_{k}= x_{k}− x^{∗} be
the error at the k-th step. Then

e_{k+1} = x_{k+1}− x^{∗}

= x_{k}− f (x_{k}) x_{k}− x_{k−1}

f (x_{k}) − f (x_{k−1}) − x^{∗}

= 1

f (x_{k}) − f (x_{k−1})[(xk−1− x^{∗})f (xk) − (xk− x^{∗})f (xk−1)]

= 1

f (x_{k}) − f (x_{k−1})(ek−1f (xk) − ekf (xk−1))

= e_{k}e_{k−1}

1

ekf (x_{k}) − _{e}^{1}

k−1f (x_{k−1})

xk− x_{k−1} · x_{k}− x_{k−1}
f (xk) − f (xk−1)

!

師大

To estimate the numerator

1

ekf (xk)− ^{1}

ek−1f (xk−1)

xk−x_{k−1} , we apply
Taylor’s Theorem

f (x_{k}) = f (x^{∗}+ e_{k}) = f (x^{∗}) + f^{0}(x^{∗})e_{k}+1

2f^{00}(x^{∗})e^{2}_{k}+ O(e^{3}_{k}),
to get

1 ek

f (x_{k}) = f^{0}(x^{∗}) +1

2f^{00}(x^{∗})e_{k}+ O(e^{2}_{k}).

Similarly, 1

e_{k−1}f (x_{k−1}) = f^{0}(x^{∗}) +1

2f^{00}(x^{∗})e_{k−1}+ O(e^{2}_{k−1}).

Hence 1

e_{k}f (x_{k}) − 1

e_{k−1}f (x_{k−1}) ≈ 1

2(e_{k}− e_{k−1})f^{00}(x^{∗}).

Since x_{k}− x_{k−1} = e_{k}− e_{k−1}and
x_{k}− x_{k−1}

f (x ) − f (x ) → 1
f^{0}(x^{∗}),

師大

we have

e_{k+1} ≈ e_{k}e_{k−1}

1

2(e_{k}− e_{k−1})f^{00}(x^{∗})
e_{k}− e_{k−1} · 1

f^{0}(x^{∗})

!

= 1 2

f^{00}(x^{∗})
f^{0}(x^{∗})e_{k}e_{k−1}

≡ Ce_{k}ek−1. (2)

To estimate the convergence rate, we assume

|e_{k+1}| ≈ η|e_{k}|^{α},
where η > 0 and α > 0 are constants, i.e.,

|e_{k+1}|

η|e_{k}|^{α} → 1 as k → ∞.

Then |e_{k}| ≈ η|e_{k−1}|^{α}which implies |e_{k−1}| ≈ η^{−1/α}|e_{k}|^{1/α}. Hence
(2) gives

η|e_{k}|^{α}≈ C|e_{k}|η^{−1/α}|e_{k}|^{1/α} =⇒ C^{−1}η^{1+}^{α}^{1} ≈ |e_{k}|^{1−α+}^{α}^{1}.

師大

Since |e_{k}| → 0 as k → ∞, and C^{−1}η^{1+}^{α}^{1} is a nonzero constant,
1 − α + 1

α = 0 =⇒ α = 1 +√ 5

2 ≈ 1.62.

This result implies that C^{−1}η^{1+}^{α}^{1} → 1 and
η → C^{1+α}^{α} = f^{00}(x^{∗})

2f^{0}(x^{∗})

0.62

. In summary, we have shown that

|e_{k+1}| = η|e_{k}|^{α}, α ≈ 1.62,
that is, therate of convergenceissuperlinear.

**Rate of convergence**

secantmethod: superlinear Newton’smethod: quadratic bisectionmethod: linear

師大

Each iteration of method requires

secant method: one function evaluation

Newton’s method: two function evaluation, namely, f (x_{k})
and f^{0}(x_{k}).

⇒ two steps of secant method are comparable to one step of Newton’s method. Thus

|e_{k+2}| ≈ η|e_{k+1}|^{α} ≈ η^{1+α}|e_{k}|^{3+}

√ 5

2 ≈ η^{1+α}|e_{k}|^{2.62}.

⇒ secant method is more efficient than Newton’s method.

**Remark**

Two steps of secant method would require a little more work than one step of Newton’s method.

師大

**Aitken’s ∆**^{2} **method**

Accelerate the convergence of a sequence that islinearly convergent.

Suppose {y_{n}}^{∞}_{n=0}is a linearly convergent sequence with
limit y. Construct a sequence {ˆy_{n}}^{∞}_{n=0}that converges more
rapidly to y than {y_{n}}^{∞}_{n=0}.

For n sufficiently large,
y_{n+1}− y

y_{n}− y ≈ y_{n+2}− y
y_{n+1}− y.
Then

(y_{n+1}− y)^{2}≈ (y_{n+2}− y)(y_{n}− y),
so

y^{2}_{n+1}− 2y_{n+1}y + y^{2} ≈ y_{n+2}yn− (y_{n+2}+ yn)y + y^{2}

師大

and

(yn+2+ yn− 2y_{n+1})y ≈ yn+2yn− y^{2}_{n+1}.
Solving for y gives

y ≈ yn+2yn− y_{n+1}^{2}
y_{n+2}− 2y_{n+1}+ y_{n}

= y_{n}y_{n+2}− 2y_{n}y_{n+1}+ y^{2}_{n}− y_{n}^{2}+ 2y_{n}y_{n+1}− y^{2}_{n+1}
yn+2− 2y_{n+1}+ yn

= y_{n}(y_{n+2}− 2y_{n+1}+ y_{n}) − (y_{n+1}− y_{n})^{2}
(y_{n+2}− y_{n+1}) − (y_{n+1}− y_{n})

= y_{n}− (y_{n+1}− y_{n})^{2}

(y_{n+2}− y_{n+1}) − (y_{n+1}− y_{n}).
**Aitken’s ∆**^{2} **method**

ˆ

y_{n}= y_{n}− (y_{n+1}− y_{n})^{2}

(yn+2− y_{n+1}) − (yn+1− y_{n}). (3)

師大

**Example 17**

The sequence {y_{n}= cos(1/n)}^{∞}_{n=1}converges linearly to y = 1.

n yn yˆn

1 0.54030 0.96178 2 0.87758 0.98213 3 0.94496 0.98979 4 0.96891 0.99342 5 0.98007 0.99541 6 0.98614

7 0.98981

{ˆyn}^{∞}_{n=1}converges more rapidly to y = 1 than {yn}^{∞}_{n=1}.

師大

**Definition 18**

For a given sequence {y_{n}}^{∞}_{n=0}, the forward difference ∆y_{n}is
defined by

∆yn= yn+1− y_{n}, for n ≥ 0.

Higher powers of ∆ are defined recursively by

∆^{k}y_{n}= ∆(∆^{k−1}y_{n}), for k ≥ 2.

The definition implies that

∆^{2}yn= ∆(yn+1− y_{n}) = ∆yn+1− ∆y_{n}= (yn+2− y_{n+1}) − (yn+1− y_{n}).

So the formula for ˆynin (3) can be written as ˆ

yn= yn−(∆y_{n})^{2}

∆^{2}yn

, for n ≥ 0.

師大

**Theorem 19**

Suppose {y_{n}}^{∞}_{n=0} → ylinearlyand

n→∞lim

yn+1− y
y_{n}− y < 1.

Then {ˆyn}^{∞}_{n=0}→ y faster than {yn}^{∞}_{n=0}in the sense that

n→∞lim
ˆ
y_{n}− y
yn− y = 0.

Aitken’s ∆^{2}method constructs the terms in order:

y0, y1 = g(y0), y2= g(y1), yˆ0= {∆^{2}}(y_{0}), y3 = g(y2),
ˆ

y1 = {∆^{2}}(y_{1}), . . . .

⇒ Assume |ˆy0− y| < |y_{2}− y|

師大

Steffensen’s method constructs the terms in order:

y_{0}^{(0)}≡ y_{0}, y_{1}^{(0)} = g(y^{(0)}_{0} ), y_{2}^{(0)}= g(y_{1}^{(0)}),
y_{0}^{(1)}= {∆^{2}}(y_{0}^{(0)}), y^{(1)}_{1} = g(y^{(1)}_{0} ), y_{2}^{(1)}= g(y^{(1)}_{1} ), . . . .
**Steffensen’s method (To find a solution of y = g(y))**

Given y_{0}, tolerance T ol, max. number of iteration M . Set i = 1.

While i ≤ M

Set y1= g(y0); y2= g(y1); y = y0− (y_{1}− y_{0})^{2}/(y2− 2y_{1}+ y0).

If |y − y_{0}| < T ol, then STOP.

Set i = i + 1; y_{0} = y.

End While
**Theorem 20**

Suppose x = g(x) has solution x^{∗} with g^{0}(x^{∗}) 6= 1. If ∃ δ > 0
such that g ∈ C^{3}[x^{∗}− δ, x^{∗}+ δ], then Steffensen’s method gives
quadraticconvergence for any x_{0} ∈ [x^{∗}− δ, x^{∗}+ δ].

師大

**Zeros of polynomials and M ¨** **uller’s method**

• Horner’s method:

Let

P (x) = a_{0}+ a_{1}x + a_{2}x^{2}+ · · · + a_{n−1}x^{n−1}+ a_{n}x^{n}

= a_{0}+ x (a_{1}+ x (a_{2}+ · · · + x (a_{n−1}+ a_{n}x) · · · )) .
If

b_{n} = a_{n},

b_{k} = a_{k}+ b_{k+1}x0, for k = n − 1, n − 2, . . . , 1, 0,
then

b0= a0+ b1x0= a0+ (a1+ b2x0) x0 = · · · = P (x0).

Consider

Q(x) = b1+ b2x + · · · + bnx^{n−1}.

師大

Then

b0+ (x − x0)Q(x) = b0+ (x − x0) b1+ b2x + · · · + bnx^{n−1}

= (b0− b_{1}x0) + (b1− b_{2}x0)x + · · · + (bn−1− b_{n}x0)x^{n−1}+ bnx^{n}

= a0+ a1x + · · · + anx^{n}= P (x).

Differentiating P (x) with respect to x gives

P^{0}(x) = Q(x) + (x − x_{0})Q^{0}(x) and P^{0}(x_{0}) = Q(x_{0}).

Use Newton-Raphson method to find an approximate zero of P (x):

xk+1= xk−P (xk)

Q(x_{k}), ∀ k = 0, 1, 2, . . . .
Similarly, let

cn = bn= an,

ck = bk+ ck+1xk, for k = n − 1, n − 2, . . . , 1,
then c_{1} = Q(x_{k}).

師大

**Horner’s method(Evaluate y = P (x**_{0})**and z = P**^{0}(x_{0}))
Set y = a_{n}; z = a_{n}.

For j = n − 1, n − 2, . . . , 1
Set y = a_{j}+ yx_{0}; z = y + zx_{0}.
End for

Set y = a_{0}+ yx0.
If x_{N} is an approximate zero of P , then

P (x) = (x − x_{N})Q(x) + b_{0} = (x − x_{N})Q(x) + P (x_{N})

≈ (x − x_{N})Q(x) ≡ (x − ˆx1)Q1(x).

So x − ˆx1is an approximate factor of P (x) and we can find a
second approximate zero of P by applying Newton’s method to
Q_{1}(x). The procedure is called deflation.

師大

• M ¨uller’s method for complex root:

**Theorem 21**

If z = a + ib is a complex zero of multiplicity m of P (x) with real
coefficients, then ¯z = a − biis also a zero of multiplicity m of
P (x)and (x^{2}− 2ax + a^{2}+ b^{2})^{m} is a factor of P (x).

Secant method: Given p_{0} and
p_{1}, determine p_{2} as the
intersection of the x-axis with
the line through (p_{0}, f (p_{0}))and
(p_{1}, f (p_{1})).

M ¨uller’s method: Given p0, p1

and p_{2}, determine p_{3} by the
intersection of the x-axis with
the parabola through (p0, f (p0)),
(p_{1}, f (p_{1}))and (p_{2}, f (p_{2})).

師大

Let

P (x) = a(x − p2)^{2}+ b(x − p2) + c

that passes through (p_{0}, f (p_{0})), (p_{1}, f (p_{1}))and (p_{2}, f (p_{2})). Then
f (p0) = a(p0− p_{2})^{2}+ b(p0− p_{2}) + c,

f (p1) = a(p1− p_{2})^{2}+ b(p1− p_{2}) + c,
f (p2) = a(p2− p_{2})^{2}+ b(p2− p_{2}) + c = c.

It implies that c = f (p2),

b = (p0− p_{2})^{2}[f (p1) − f (p2)] − (p1− p_{2})^{2}[f (p0) − f (p2)]

(p0− p_{2})(p1− p_{2})(p0− p_{1}) ,
a = (p_{1}− p_{2}) [f (p_{0}) − f (p_{2})] − (p_{0}− p_{2}) [f (p_{1}) − f (p_{2})]

(p_{0}− p_{2})(p_{1}− p_{2})(p_{0}− p_{1}) .

師大

To determine p_{3}, a zero of P , we apply the quadratic formula to
P (x) = 0and get

p_{3}− p_{2} = 2c
b ±√

b^{2}− 4ac.
Choose

p_{3} = p_{2}+ 2c
b + sgn(b)√

b^{2}− 4ac

such that the denominator will be largest and result in p_{3}
selected as the closest zero of P to p2.

師大

**M ¨uller’s method (Find a solution of f (x) = 0)**

Given p_{0}, p_{1}, p_{2}; tolerance T OL; maximum number of iterations M
Set h_{1} = p_{1}− p_{0}; h_{2} = p_{2}− p_{1};

δ1= (f (p1) − f (p0))/h1; δ_{2} = (f (p2) − f (p1))/h2;
d = (δ2− δ_{1})/(h2+ h1); i = 3.

While i ≤ M

Set b = δ_{2}+ h2d; D =pb^{2}− 4f (p_{2})d.

If |b − D| < |b + D|, then set E = b + D else set E = b − D.

Set h = −2f (p_{2})/E; p = p_{2}+ h.

If |h| < T OL, then STOP.

Set p_{0}= p_{1}; p_{1} = p_{2}; p_{2} = p; h_{1}= p_{1}− p_{0}; h_{2}= p_{2}− p_{1};
δ_{1}= (f (p_{1}) − f (p_{0}))/h_{1}; δ_{2} = (f (p_{2}) − f (p_{1}))/h_{2};
d = (δ2− δ_{1})/(h2+ h1); i = i + 1.

End while