師大

Tsung-Ming Huang

Department of Mathematics National Taiwan Normal University, Taiwan

October 13, 2014

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Outline**

**1** **Bisection Method**

**2** **Fixed-Point Iteration**

**3** **Newton’s method**

**4** **Error analysis for iterative methods**

**5** **Accelerating convergence**

**6** **Zeros of polynomials and M ¨uller’s method**

師大

**Bisection Method**

**Idea**

Iff (x) ∈ C[a, b]andf (a)f (b) < 0, then∃ c ∈ (a, b)such that f (c) = 0.

師大

**Bisection method algorithm**

Given f (x) defined on (a, b), the maximal number of iterations M, and stop criteria δ and ε, this algorithm tries to locate one root of f (x).

Compute u = f (a), v = f (b), and e = b − a
**If sign(u) = sign(v), then stop**

**For k = 1, 2, . . . , M**

e = e/2, c = a + e, w = f (c)
**If |e| < δ or |w| < ε, then stop**
**If sign(w) 6= sign(u)**

b = c, v = w
**Else**

a = c, u = w
**End If**

**End For**

師大

Let {c_{n}} be the sequence of numbers produced. The algorithm
should stop if one of the following conditions is satisfied.

**1** the iteration number k > M ,

**2** |c_{k}− c_{k−1}| < δ, or

**3** |f (c_{k})| < ε.

Let [a_{0}, b_{0}], [a_{1}, b_{1}], . . .denote the successive intervals produced
by the bisection algorithm. Then

a = a_{0}≤ a_{1} ≤ a_{2} ≤ · · · ≤ b_{0}= b

⇒ {a_{n}} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

師大

Let {c_{n}} be the sequence of numbers produced. The algorithm
should stop if one of the following conditions is satisfied.

**1** the iteration number k > M ,

**2** |c_{k}− c_{k−1}| < δ, or

**3** |f (c_{k})| < ε.

Let [a_{0}, b_{0}], [a_{1}, b_{1}], . . .denote the successive intervals produced
by the bisection algorithm.Then

a = a_{0}≤ a_{1} ≤ a_{2} ≤ · · · ≤ b_{0}= b

⇒ {a_{n}} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

師大

Let {c_{n}} be the sequence of numbers produced. The algorithm
should stop if one of the following conditions is satisfied.

**1** the iteration number k > M ,

**2** |c_{k}− c_{k−1}| < δ, or

**3** |f (c_{k})| < ε.

Let [a_{0}, b_{0}], [a_{1}, b_{1}], . . .denote the successive intervals produced
by the bisection algorithm. Then

a = a_{0}≤ a_{1} ≤ a_{2} ≤ · · · ≤ b_{0}= b

⇒ {a_{n}} and {bn} are bounded

⇒ lim

n→∞an and lim

n→∞bn exist

師大

Since

b1− a_{1} = 1

2(b0− a_{0})
b2− a_{2} = 1

2(b1− a_{1}) = 1

4(b0− a_{0})
...

b_{n}− a_{n} = 1

2^{n}(b_{0}− a_{0})
hence

n→∞lim b_{n}− lim

n→∞a_{n}= lim

n→∞(b_{n}− a_{n}) = lim

n→∞

1

2^{n}(b_{0}− a_{0}) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

師大

Since

b1− a_{1} = 1

2(b0− a_{0})
b2− a_{2} = 1

2(b1− a_{1}) = 1

4(b0− a_{0})
...

b_{n}− a_{n} = 1

2^{n}(b_{0}− a_{0})
hence

n→∞lim b_{n}− lim

n→∞a_{n}= lim

n→∞(b_{n}− a_{n}) = lim

n→∞

1

2^{n}(b_{0}− a_{0}) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

師大

Since

b1− a_{1} = 1

2(b0− a_{0})
b2− a_{2} = 1

2(b1− a_{1}) = 1

4(b0− a_{0})
...

b_{n}− a_{n} = 1

2^{n}(b_{0}− a_{0})
hence

n→∞lim b_{n}− lim

n→∞a_{n}= lim

n→∞(b_{n}− a_{n}) = lim

n→∞

1

2^{n}(b_{0}− a_{0}) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

師大

Since

b1− a_{1} = 1

2(b0− a_{0})
b2− a_{2} = 1

2(b1− a_{1}) = 1

4(b0− a_{0})
...

b_{n}− a_{n} = 1

2^{n}(b_{0}− a_{0})
hence

n→∞lim b_{n}− lim

n→∞a_{n}= lim

n→∞(b_{n}− a_{n}) = lim

n→∞

1

2^{n}(b_{0}− a_{0}) = 0.

Therefore

n→∞lim an= lim

n→∞bn≡ z.

Since f is a continuous function, we have that

n→∞lim f (an) = f ( lim

n→∞an) = f (z) and lim

n→∞f (bn) = f ( lim

n→∞bn) = f (z).

師大

On the other hand,

f (a_{n})f (b_{n}) < 0

⇒ lim

n→∞f (a_{n})f (b_{n}) = f^{2}(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {a_{n}} and {b_{n}} is a zero of
f in [a, b]. Let c_{n}= ^{1}_{2}(a_{n}+ b_{n}).Then

|z − c_{n}| =
lim

n→∞a_{n}−1

2(a_{n}+ b_{n})

= 1 2

h

n→∞lim a_{n}− b_{n}i
+ 1

2 h

n→∞lim a_{n}− a_{n}i

≤ max lim

n→∞an− b_{n}
,

lim

n→∞an− a_{n}

≤ |b_{n}− a_{n}| = 1

2^{n}|b_{0}− a_{0}|.

This proves the following theorem.

師大

On the other hand,

f (a_{n})f (b_{n}) < 0

⇒ lim

n→∞f (a_{n})f (b_{n}) = f^{2}(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {a_{n}} and {b_{n}} is a zero of
f in [a, b].Let c_{n}= ^{1}_{2}(a_{n}+ b_{n}). Then

|z − c_{n}| =
lim

n→∞a_{n}−1

2(a_{n}+ b_{n})

= 1 2

h

n→∞lim a_{n}− b_{n}i
+ 1

2 h

n→∞lim a_{n}− a_{n}i

≤ max lim

n→∞an− b_{n}
,

lim

n→∞an− a_{n}

≤ |b_{n}− a_{n}| = 1

2^{n}|b_{0}− a_{0}|.

This proves the following theorem.

師大

On the other hand,

f (a_{n})f (b_{n}) < 0

⇒ lim

n→∞f (a_{n})f (b_{n}) = f^{2}(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {a_{n}} and {b_{n}} is a zero of
f in [a, b]. Let c_{n}= ^{1}_{2}(a_{n}+ b_{n}).Then

|z − c_{n}| =
lim

n→∞a_{n}−1

2(a_{n}+ b_{n})

= 1 2

h

n→∞lim a_{n}− b_{n}i
+ 1

2 h

n→∞lim a_{n}− a_{n}i

≤ max lim

n→∞an− b_{n}
,

lim

n→∞an− a_{n}

≤ |b_{n}− a_{n}| = 1

2^{n}|b_{0}− a_{0}|.

This proves the following theorem.

師大

On the other hand,

f (a_{n})f (b_{n}) < 0

⇒ lim

n→∞f (a_{n})f (b_{n}) = f^{2}(z) ≤ 0

⇒ f (z) = 0

Therefore, the limit of the sequences {a_{n}} and {b_{n}} is a zero of
f in [a, b]. Let c_{n}= ^{1}_{2}(a_{n}+ b_{n}). Then

|z − c_{n}| =
lim

n→∞a_{n}−1

2(a_{n}+ b_{n})

= 1 2

h

n→∞lim a_{n}− b_{n}i
+ 1

2 h

n→∞lim a_{n}− a_{n}i

≤ max lim

n→∞an− b_{n}
,

lim

n→∞an− a_{n}

≤ |b_{n}− a_{n}| = 1

2^{n}|b_{0}− a_{0}|.

This proves the following theorem.

師大

**Theorem 1**

Let{[a_{n}, bn]}denote the intervals produced by the bisection
algorithm. Then lim

n→∞anand lim

n→∞bnexist, areequal, and represent azerooff (x). If

z = lim

n→∞an= lim

n→∞bn and cn= 1

2(an+ bn), then

|z − c_{n}| ≤ 1

2^{n}(b_{0}− a_{0}) .
**Remark**

{c_{n}}convergestozwith therateofO(2^{−n}).

師大

**Theorem 1**

Let{[a_{n}, bn]}denote the intervals produced by the bisection
algorithm. Then lim

n→∞anand lim

n→∞bnexist, areequal, and represent azerooff (x). If

z = lim

n→∞an= lim

n→∞bn and cn= 1

2(an+ bn), then

|z − c_{n}| ≤ 1

2^{n}(b_{0}− a_{0}) .
**Remark**

{c_{n}}convergestozwith therateofO(2^{−n}).

師大

**Example 2**

How many steps should be taken to compute a root of
f (x) = x^{3}+ 4x^{2}− 10 = 0 on [1, 2] with relative error 10^{−3}?
solution: Seek an n such that

|z − c_{n}|

|z| ≤ 10^{−3} ⇒ |z − c_{n}| ≤ |z| × 10^{−3}.
Since z ∈ [1, 2], it is sufficient to show

|z − c_{n}| ≤ 10^{−3}.
That is, we solve

2^{−n}(2 − 1) ≤ 10^{−3} ⇒ −n log_{10}2 ≤ −3
which gives n ≥ 10.

師大

**Example 2**

How many steps should be taken to compute a root of
f (x) = x^{3}+ 4x^{2}− 10 = 0 on [1, 2] with relative error 10^{−3}?
solution: Seek an n such that

|z − c_{n}|

|z| ≤ 10^{−3} ⇒ |z − c_{n}| ≤ |z| × 10^{−3}.
Since z ∈ [1, 2], it is sufficient to show

|z − c_{n}| ≤ 10^{−3}.
That is, we solve

2^{−n}(2 − 1) ≤ 10^{−3} ⇒ −n log_{10}2 ≤ −3
which gives n ≥ 10.

師大

**Exercise**

Page 54: 1, 13, 14, 16, 17

師大

**Fixed-Point Iteration**

**Definition 3**

xis called afixed pointof a given functiongifg(x) = x.

**Root-finding problems and fixed-point problems**
Find x^{∗} such that f (x^{∗}) = 0.

Let g(x) = x − f (x). Then g(x^{∗}) = x^{∗}− f (x^{∗}) = x^{∗}.

⇒ x^{∗}is a fixed point for g(x).

Find x^{∗} such that g(x^{∗}) = x^{∗}.
Define f (x) = x − g(x) so that
f (x^{∗}) = x^{∗}− g(x^{∗}) = x^{∗}− x^{∗}= 0

⇒ x^{∗}is a zero of f (x).

師大

**Fixed-Point Iteration**

**Definition 3**

xis called afixed pointof a given functiongifg(x) = x.

**Root-finding problems and fixed-point problems**
Find x^{∗} such that f (x^{∗}) = 0.

Let g(x) = x − f (x). Then g(x^{∗}) = x^{∗}− f (x^{∗}) = x^{∗}.

⇒ x^{∗}is a fixed point for g(x).

Find x^{∗} such that g(x^{∗}) = x^{∗}.
Define f (x) = x − g(x) so that
f (x^{∗}) = x^{∗}− g(x^{∗}) = x^{∗}− x^{∗}= 0

⇒ x^{∗}is a zero of f (x).

師大

**Fixed-Point Iteration**

**Definition 3**

xis called afixed pointof a given functiongifg(x) = x.

**Root-finding problems and fixed-point problems**
Find x^{∗} such that f (x^{∗}) = 0.

Let g(x) = x − f (x). Then g(x^{∗}) = x^{∗}− f (x^{∗}) = x^{∗}.

⇒ x^{∗}is a fixed point for g(x).

Find x^{∗} such that g(x^{∗}) = x^{∗}.
Define f (x) = x − g(x) so that
f (x^{∗}) = x^{∗}− g(x^{∗}) = x^{∗}− x^{∗}= 0

⇒ x^{∗}is a zero of f (x).

師大

**Example 4**

The function g(x) = x^{2}− 2, for −2 ≤ x ≤ 3, has fixed points at
x = −1and x = 2 since

0 = g(x) − x = x^{2}− x − 2 = (x + 1)(x − 2).

師大

**Theorem 5 (Existence and uniqueness)**

**1** If g ∈ C[a, b] such thata ≤ g(x) ≤ bfor all x ∈ [a, b], then g
hasa fixed point in [a, b].

**2** If, in addition, g^{0}(x)exists in (a, b) and there exists a
positive constantM < 1such that|g^{0}(x)| ≤ M < 1for all
x ∈ (a, b). Then the fixed point isunique.

師大

**Theorem 5 (Existence and uniqueness)**

**1** If g ∈ C[a, b] such thata ≤ g(x) ≤ bfor all x ∈ [a, b], then g
hasa fixed point in [a, b].

**2** If, in addition, g^{0}(x)exists in (a, b) and there exists a
positive constantM < 1such that|g^{0}(x)| ≤ M < 1for all
x ∈ (a, b). Then the fixed point isunique.

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0. That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0.That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

Otherwise, it must be g(a) > a and g(b) < b. The function h(x) = g(x) − xis continuous on [a, b], with

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0. That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0.That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Existence:

If g(a) = a or g(b) = b, then a or b is a fixed point of g and we are done.

h(a) = g(a) − a > 0 and h(b) = g(b) − b < 0.

By the Intermediate Value Theorem, ∃ x^{∗} ∈ [a, b] such that
h(x^{∗}) = 0. That is

g(x^{∗}) − x^{∗} = 0 ⇒ g(x^{∗}) = x^{∗}.
Hence g has a fixed point x^{∗}in [a, b].

師大

**Proof**

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b].By the Mean-Value theorem, there exists ξ between p and q such that

g^{0}(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g^{0}(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is
unique.

師大

**Proof**

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b]. By the Mean-Value theorem, there exists ξ between p and q such that

g^{0}(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g^{0}(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is
unique.

師大

**Proof**

Uniqueness:

Suppose that p 6= q are both fixed points of g in [a, b]. By the Mean-Value theorem, there exists ξ between p and q such that

g^{0}(ξ) = g(p) − g(q)

p − q = p − q p − q = 1.

However, this contradicts to the assumption that

|g^{0}(x)| ≤ M < 1for all x in [a, b]. Therefore the fixed point of g is
unique.

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

**Example 6**

Show that the following function has a unique fixed point.

g(x) = (x^{2}− 1)/3, x ∈ [−1, 1].

Solution: The Extreme Value Theorem implies that min

x∈[−1,1]g(x) = g(0) = −1 3, max

x∈[−1,1]g(x) = g(±1) = 0.

That is g(x) ∈ [−1, 1], ∀ x ∈ [−1, 1].

Moreover, g is continuous and

|g^{0}(x)| =

2x 3

≤ 2

3, ∀ x ∈ (−1, 1).

By above theorem, g has a unique fixed point in [−1, 1].

師大

Let p be such unique fixed point of g. Then
p = g(p) = p^{2}− 1

3 ⇒ p^{2}− 3p − 1 = 0

⇒ p = 1 2(3 −√

13).

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x. However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x.However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x. However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x. However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration or functional iteration**

Given a continuous function g, choose an initial point x_{0} and
generate {x_{k}}^{∞}_{k=0}by

x_{k+1} = g(x_{k}), k ≥ 0.

{x_{k}} may not converge, e.g., g(x) = 3x. However, when the
sequence converges, say,

k→∞lim x_{k}= x^{∗},
then, since g is continuous,

g(x^{∗}) = g( lim

k→∞x_{k}) = lim

k→∞g(x_{k}) = lim

k→∞x_{k+1}= x^{∗}.
That is, x^{∗}is a fixed point of g.

師大

**Fixed-point iteration**

Given x_{0}, tolerance T OL, maximum number of iteration M .
Set i = 1 and x = g(x_{0}).

While i ≤ M and |x − x0| ≥ T OL
Set i = i + 1, x_{0} = xand x = g(x_{0}).

End While

師大

**Example 7**
The equation

x^{3}+ 4x^{2}− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g_{1}(x) ≡ x − f (x) = x − x^{3}− 4x^{2}+ 10

(b) x = g_{2}(x) = ^{10}_{x} − 4x1/2

x^{3} = 10 − 4x^{2} ⇒ x^{2} = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

師大

**Example 7**
The equation

x^{3}+ 4x^{2}− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g_{1}(x) ≡ x − f (x) = x − x^{3}− 4x^{2}+ 10

(b) x = g_{2}(x) = ^{10}_{x} − 4x1/2

x^{3} = 10 − 4x^{2} ⇒ x^{2} = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

師大

**Example 7**
The equation

x^{3}+ 4x^{2}− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g_{1}(x) ≡ x − f (x) = x − x^{3}− 4x^{2}+ 10

(b) x = g_{2}(x) = ^{10}_{x} − 4x1/2

x^{3} = 10 − 4x^{2} ⇒ x^{2} = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

師大

**Example 7**
The equation

x^{3}+ 4x^{2}− 10 = 0

has a unique root in [1, 2]. Change the equation to the fixed-point form x = g(x).

(a) x = g_{1}(x) ≡ x − f (x) = x − x^{3}− 4x^{2}+ 10

(b) x = g_{2}(x) = ^{10}_{x} − 4x1/2

x^{3} = 10 − 4x^{2} ⇒ x^{2} = 10

x − 4x ⇒ x = ± 10 x − 4x

1/2

師大

(c) x = g3(x) = ^{1}_{2} 10 − x^{3}1/2

4x^{2} = 10 − x^{3} ⇒ x = ±1

2 10 − x^{3}1/2

(d) x = g_{4}(x) =

10 4+x

1/2

x^{2}(x + 4) = 10 ⇒ x = ±

10 4 + x

1/2

(e) x = g_{5}(x) = x −^{x}^{3}_{3x}^{+4x}2+8x^{2}^{−10}

x = g5(x) ≡ x − f (x)
f^{0}(x)

師大

(c) x = g3(x) = ^{1}_{2} 10 − x^{3}1/2

4x^{2} = 10 − x^{3} ⇒ x = ±1

2 10 − x^{3}1/2

(d) x = g_{4}(x) =

10 4+x

1/2

x^{2}(x + 4) = 10 ⇒ x = ±

10 4 + x

1/2

(e) x = g_{5}(x) = x −^{x}^{3}_{3x}^{+4x}2+8x^{2}^{−10}

x = g5(x) ≡ x − f (x)
f^{0}(x)

師大

(c) x = g3(x) = ^{1}_{2} 10 − x^{3}1/2

4x^{2} = 10 − x^{3} ⇒ x = ±1

2 10 − x^{3}1/2

(d) x = g_{4}(x) =

10 4+x

1/2

x^{2}(x + 4) = 10 ⇒ x = ±

10 4 + x

1/2

(e) x = g_{5}(x) = x −^{x}^{3}_{3x}^{+4x}2+8x^{2}^{−10}

x = g5(x) ≡ x − f (x)
f^{0}(x)

師大

Results of the fixed-point iteration with initial point x_{0} = 1.5

師大

**Theorem 8 (Fixed-point Theorem)**

Let g ∈ C[a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b].

Suppose that g^{0} exists on (a, b) and that ∃ k with0 < k < 1such
that

|g^{0}(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x_{0} in [a, b],

x_{n}= g(x_{n−1}), n ≥ 1,
converges to the unique fixed point x in [a, b].

師大

**Theorem 8 (Fixed-point Theorem)**

Let g ∈ C[a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b].

Suppose that g^{0} exists on (a, b) and that ∃ k with0 < k < 1such
that

|g^{0}(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x_{0} in [a, b],

x_{n}= g(x_{n−1}), n ≥ 1,
converges to the unique fixed point x in [a, b].

師大

**Theorem 8 (Fixed-point Theorem)**

Let g ∈ C[a, b] be such thatg(x) ∈ [a, b]for all x ∈ [a, b].

Suppose that g^{0} exists on (a, b) and that ∃ k with0 < k < 1such
that

|g^{0}(x)| ≤ k, ∀ x ∈ (a, b).

Then, for any number x_{0} in [a, b],

x_{n}= g(x_{n−1}), n ≥ 1,
converges to the unique fixed point x in [a, b].

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {x_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b).It follows that

|x_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞} converges to x.

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {x_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b). It follows that

|x_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞} converges to x.

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

Since g([a, b]) ⊆ [a, b], {x_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b).It follows that

|x_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞} converges to x.

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b). It follows that

_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞} converges to x.

師大

Proof: By the assumptions, a unique fixed point exists in [a, b].

_{n}}^{∞}_{n=0} is defined and x_{n}∈ [a, b] for all
n ≥ 0. Using the Mean Values Theorem and the fact that

|g^{0}(x)| ≤ k, we have

|x − x_{n}| = |g(x_{n−1}) − g(x)| = |g^{0}(ξn)||x − xn−1| ≤ k|x − x_{n−1}|,
where ξn∈ (a, b). It follows that

_{n}− x| ≤ k|x_{n−1}− x| ≤ k^{2}|x_{n−2}− x| ≤ · · · ≤ k^{n}|x_{0}− x|. (1)
Since 0 < k < 1, we have

n→∞lim k^{n}= 0
and

n→∞lim |x_{n}− x| ≤ lim

n→∞k^{n}|x_{0}− x| = 0.

Hence, {x_{n}}^{∞} converges to x.

師大

**Corollary 9**

If g satisfies the hypotheses of above theorem, then

|x − x_{n}| ≤ k^{n}max{x_{0}− a, b − x_{0}}
and

|x_{n}− x| ≤ k^{n}

1 − k|x_{1}− x_{0}|, ∀ n ≥ 1.

Proof: From (1),

|x_{n}− x| ≤ k^{n}|x_{0}− x| ≤ k^{n}max{x_{0}− a, b − x_{0}}.

For n ≥ 1, using the Mean Values Theorem,

|x_{n+1}− x_{n}| = |g(x_{n}) − g(x_{n−1})| ≤ k|x_{n}− x_{n−1}| ≤ · · · ≤ k^{n}|x_{1}− x_{0}|.

師大

**Corollary 9**

If g satisfies the hypotheses of above theorem, then

|x − x_{n}| ≤ k^{n}max{x_{0}− a, b − x_{0}}
and

|x_{n}− x| ≤ k^{n}

1 − k|x_{1}− x_{0}|, ∀ n ≥ 1.

Proof: From (1),

|x_{n}− x| ≤ k^{n}|x_{0}− x| ≤ k^{n}max{x_{0}− a, b − x_{0}}.

For n ≥ 1, using the Mean Values Theorem,

|x_{n+1}− x_{n}| = |g(x_{n}) − g(x_{n−1})| ≤ k|x_{n}− x_{n−1}| ≤ · · · ≤ k^{n}|x_{1}− x_{0}|.

師大

**Corollary 9**

If g satisfies the hypotheses of above theorem, then

|x − x_{n}| ≤ k^{n}max{x_{0}− a, b − x_{0}}
and

|x_{n}− x| ≤ k^{n}

1 − k|x_{1}− x_{0}|, ∀ n ≥ 1.

Proof: From (1),

|x_{n}− x| ≤ k^{n}|x_{0}− x| ≤ k^{n}max{x_{0}− a, b − x_{0}}.

For n ≥ 1, using the Mean Values Theorem,

|x_{n+1}− x_{n}| = |g(x_{n}) − g(x_{n−1})| ≤ k|x_{n}− x_{n−1}| ≤ · · · ≤ k^{n}|x_{1}− x_{0}|.

師大

Thus, for m > n ≥ 1,

|x_{m}− x_{n}| = |x_{m}− x_{m−1}+ xm−1− · · · + x_{n+1}− x_{n}|

≤ |x_{m}− x_{m−1}| + |x_{m−1}− x_{m−2}| + · · · + |x_{n+1}− x_{n}|

≤ k^{m−1}|x_{1}− x_{0}| + k^{m−2}|x_{1}− x_{0}| + · · · + k^{n}|x_{1}− x_{0}|

= k^{n}|x_{1}− x_{0}| 1 + k + k^{2}+ · · · + k^{m−n−1} .
It implies that

|x − x_{n}| = lim

m→∞|x_{m}− x_{n}| ≤ lim

m→∞k^{n}|x_{1}− x_{0}|

m−n−1

X

j=0

k^{j}

≤ k^{n}|x_{1}− x_{0}|

∞

X

j=0

k^{j} = k^{n}

1 − k|x_{1}− x_{0}|.

師大

Thus, for m > n ≥ 1,

|x_{m}− x_{n}| = |x_{m}− x_{m−1}+ xm−1− · · · + x_{n+1}− x_{n}|

≤ |x_{m}− x_{m−1}| + |x_{m−1}− x_{m−2}| + · · · + |x_{n+1}− x_{n}|

≤ k^{m−1}|x_{1}− x_{0}| + k^{m−2}|x_{1}− x_{0}| + · · · + k^{n}|x_{1}− x_{0}|

= k^{n}|x_{1}− x_{0}| 1 + k + k^{2}+ · · · + k^{m−n−1} .
It implies that

|x − x_{n}| = lim

m→∞|x_{m}− x_{n}| ≤ lim

m→∞k^{n}|x_{1}− x_{0}|

m−n−1

X

j=0

k^{j}

≤ k^{n}|x_{1}− x_{0}|

∞

X

j=0

k^{j} = k^{n}

1 − k|x_{1}− x_{0}|.

師大

**Example 10**

For previous example,

f (x) = x^{3}+ 4x^{2}− 10 = 0.

For g_{1}(x) = x − x^{3}− 4x^{2}+ 10, we have
g_{1}(1) = 6 and g_{1}(2) = −12,
so g_{1}([1, 2]) * [1, 2].Moreover,

g_{1}^{0}(x) = 1 − 3x^{2}− 8x ⇒ |g_{1}^{0}(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

師大

**Example 10**

For previous example,

f (x) = x^{3}+ 4x^{2}− 10 = 0.

For g_{1}(x) = x − x^{3}− 4x^{2}+ 10, we have
g_{1}(1) = 6 and g_{1}(2) = −12,
so g_{1}([1, 2]) * [1, 2].Moreover,

g_{1}^{0}(x) = 1 − 3x^{2}− 8x ⇒ |g_{1}^{0}(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

師大

**Example 10**

For previous example,

f (x) = x^{3}+ 4x^{2}− 10 = 0.

For g_{1}(x) = x − x^{3}− 4x^{2}+ 10, we have
g_{1}(1) = 6 and g_{1}(2) = −12,
so g_{1}([1, 2]) * [1, 2]. Moreover,

g_{1}^{0}(x) = 1 − 3x^{2}− 8x ⇒ |g_{1}^{0}(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

師大

**Example 10**

For previous example,

f (x) = x^{3}+ 4x^{2}− 10 = 0.

For g_{1}(x) = x − x^{3}− 4x^{2}+ 10, we have
g_{1}(1) = 6 and g_{1}(2) = −12,
so g_{1}([1, 2]) * [1, 2]. Moreover,

g_{1}^{0}(x) = 1 − 3x^{2}− 8x ⇒ |g_{1}^{0}(x)| ≥ 1 ∀ x ∈ [1, 2]

• DOES NOT guarantee to converge or not

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5]and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{3}(x) = ^{1}_{2}(10 − x^{3})^{1/2}, ∀ x ∈ [1, 1.5],
g_{3}^{0}(x) = −3

4x^{2}(10 − x^{3})^{−1/2}< 0, ∀ x ∈ [1, 1.5],
so g_{3} is strictly decreasing on [1, 1.5] and

1 < 1.28 ≈ g_{3}(1.5) ≤ g_{3}(x) ≤ g_{3}(1) = 1.5, ∀ x ∈ [1, 1.5].

On the other hand,

|g^{0}_{3}(x)| ≤ |g^{0}_{3}(1.5)| ≈ 0.66, ∀ x ∈ [1, 1.5].

Hence, the sequence is convergent to the fixed point.

師大

For g_{4}(x) =p10/(4 + x), we have
r10

6 ≤ g_{4}(x) ≤
r10

5 , ∀ x ∈ [1, 2] ⇒ g_{4}([1, 2]) ⊆ [1, 2]

Moreover,

|g^{0}_{4}(x)| =

√ −5

10(4 + x)^{3/2}

≤ 5

√10(5)^{3/2} < 0.15, ∀ x ∈ [1, 2].

The bound of |g^{0}_{4}(x)|is much smaller than the bound of |g_{3}^{0}(x)|,
which explains the more rapid convergence using g_{4}.

師大

For g_{4}(x) =p10/(4 + x), we have
r10

6 ≤ g_{4}(x) ≤
r10

5 , ∀ x ∈ [1, 2] ⇒ g_{4}([1, 2]) ⊆ [1, 2]

Moreover,

|g^{0}_{4}(x)| =

√ −5

10(4 + x)^{3/2}

≤ 5

√10(5)^{3/2} < 0.15, ∀ x ∈ [1, 2].

The bound of |g^{0}_{4}(x)|is much smaller than the bound of |g_{3}^{0}(x)|,
which explains the more rapid convergence using g_{4}.

師大

For g_{4}(x) =p10/(4 + x), we have
r10

6 ≤ g_{4}(x) ≤
r10

5 , ∀ x ∈ [1, 2] ⇒ g_{4}([1, 2]) ⊆ [1, 2]

Moreover,

|g^{0}_{4}(x)| =

√ −5

10(4 + x)^{3/2}

≤ 5

√10(5)^{3/2} < 0.15, ∀ x ∈ [1, 2].

The bound of |g^{0}_{4}(x)|is much smaller than the bound of |g_{3}^{0}(x)|,
which explains the more rapid convergence using g_{4}.

師大

**Exercise**

Page 64: 1, 3, 7, 11, 13

師大

Suppose that f : R → R and f ∈ C^{2}[a, b], i.e., f^{00}exists and is
continuous.If f (x^{∗}) = 0and x^{∗}= x + hwhere h is small, then
byTaylor’stheorem

0 = f (x^{∗}) = f (x + h)

= f (x) + f^{0}(x)h + 1

2f^{00}(x)h^{2}+ 1

3!f^{000}(x)h^{3}+ · · ·

= f (x) + f^{0}(x)h + O(h^{2}).

Sincehissmall, O(h^{2})is negligible. It is reasonable to drop
O(h^{2})terms. This implies

f (x) + f^{0}(x)h ≈ 0 and h ≈ −f (x)

f^{0}(x), if f^{0}(x) 6= 0.

Hence

x + h = x − f (x)
f^{0}(x)
is a better approximation to x^{∗}.

師大

Suppose that f : R → R and f ∈ C^{2}[a, b], i.e., f^{00}exists and is
continuous. If f (x^{∗}) = 0and x^{∗}= x + hwhere h is small,then
byTaylor’stheorem

0 = f (x^{∗}) = f (x + h)

= f (x) + f^{0}(x)h + 1

2f^{00}(x)h^{2}+ 1

3!f^{000}(x)h^{3}+ · · ·

= f (x) + f^{0}(x)h + O(h^{2}).

Sincehissmall, O(h^{2})is negligible. It is reasonable to drop
O(h^{2})terms. This implies

f (x) + f^{0}(x)h ≈ 0 and h ≈ −f (x)

f^{0}(x), if f^{0}(x) 6= 0.

Hence

x + h = x − f (x)
f^{0}(x)
is a better approximation to x^{∗}.

師大

Suppose that f : R → R and f ∈ C^{2}[a, b], i.e., f^{00}exists and is
continuous. If f (x^{∗}) = 0and x^{∗}= x + hwhere h is small, then
byTaylor’stheorem

0 = f (x^{∗}) = f (x + h)

= f (x) + f^{0}(x)h + 1

2f^{00}(x)h^{2}+ 1

3!f^{000}(x)h^{3}+ · · ·

= f (x) + f^{0}(x)h + O(h^{2}).

Sincehissmall, O(h^{2})is negligible. It is reasonable to drop
O(h^{2})terms. This implies

f (x) + f^{0}(x)h ≈ 0 and h ≈ −f (x)

f^{0}(x), if f^{0}(x) 6= 0.

Hence

x + h = x − f (x)
f^{0}(x)
is a better approximation to x^{∗}.