Sol. Non-linear Fun.

(1)

Sol. Non-linear Fun.

¹

Solutions of Non-linear Equations in One Variable

NTNU

Tsung-Min Hwang November 16, 2003

Department of Mathematics – NTNU Tsung-Min Hwang November 16, 2003

(2)

Sol. Non-linear Fun.

²

1 – Preliminaries . . . 3

2 – Bisection Method . . . 6

3 – Newton’s Method . . . 12

3.1 – Derivation of Newton’s Method . . . 12

3.2 – Convergence Analysis . . . 16

3.3 – Examples . . . 24

4 – Quasi-Newton’s Method (Secant Method) . . . 27

4.1 – The Secant Method . . . 27

4.2 – Error Analysis of Secant Method . . . 31

5 – Fixed Point and Functional Iteration . . . 36

5.1 – Functional Iteration . . . 37

5.2 – Convergence Analysis . . . 43

(3)

Sol. Non-linear Fun.

³

1 – Preliminaries

Definition 1 Let

{x

ⁿ

} → x

^∗. We say that the rate of convergence is

1. linear if

∃

^{a constant}

0 < c < 1

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|, ∀ n ≥ N;

2. superlinear if

∃ {c

ⁿ

}

^,

c

_n

→ 0

^as

n → ∞

, and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c

ⁿ

|x

ⁿ

− x

^∗

|, ∀ n ≥ N,

or, equivalently,

n→∞

lim

|x

ⁿ⁺¹

− x

^∗

|

|x

ⁿ

− x

^∗

| = 0;

3. quadratic if

∃

^{a constant}

c > 0

(not necessarily less than 1) and an integer

N > 0

such that

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

²

, ∀ n ≥ N.

(4)

Sol. Non-linear Fun.

³

1 – Preliminaries

{x

ⁿ

} → x

^∗. We say that the rate of convergence is 1. linear if

∃

^{a constant}

0 < c < 1

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|, ∀ n ≥ N;

2. superlinear if

∃ {c

ⁿ

}

^,

c

_n

→ 0

^as

n → ∞

, and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c

ⁿ

|x

ⁿ

− x

^∗

|, ∀ n ≥ N,

or, equivalently,

n→∞

lim

|x

ⁿ⁺¹

− x

^∗

|

|x

ⁿ

− x

^∗

| = 0;

3. quadratic if

∃

^{a constant}

c > 0

N > 0

such that

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

²

, ∀ n ≥ N.

(5)

Sol. Non-linear Fun.

³

1 – Preliminaries

{x

ⁿ

} → x

∃

^{a constant}

0 < c < 1

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|, ∀ n ≥ N;

2. superlinear if

∃ {c

ⁿ

}

^,

c

_n

→ 0

^as

n → ∞

, and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c

ⁿ

|x

ⁿ

− x

^∗

|, ∀ n ≥ N,

or, equivalently,

n→∞

lim

|x

ⁿ⁺¹

− x

^∗

|

|x

ⁿ

− x

^∗

| = 0;

3. quadratic if

∃

^{a constant}

c > 0

N > 0

such that

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

²

, ∀ n ≥ N.

(6)

Sol. Non-linear Fun.

³

1 – Preliminaries

{x

ⁿ

} → x

∃

^{a constant}

0 < c < 1

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|, ∀ n ≥ N;

2. superlinear if

∃ {c

ⁿ

}

^,

c

_n

→ 0

^as

n → ∞

, and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c

ⁿ

|x

ⁿ

− x

^∗

|, ∀ n ≥ N,

or, equivalently,

n→∞

lim

|x

ⁿ⁺¹

− x

^∗

|

|x

ⁿ

− x

^∗

| = 0;

3. quadratic if

∃

^{a constant}

c > 0

N > 0

such that

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

²

, ∀ n ≥ N.

(7)

Sol. Non-linear Fun.

⁴

In general, if there are positive constants

c

^and

α

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

^α

, ∀ n ≥ N,

then we say the rate of convergence is of order

α

^.

Definition 2 Suppose

{β

ⁿ

} → 0

^and

{x

ⁿ

} → x

^∗^{. If}

∃ c > 0

and an integer

N > 0

such that

|x

ⁿ

− x

^∗

| ≤ c|β

ⁿ

|, ∀ n ≥ N,

then we say

{x

ⁿ

}

^converges ^to

x

^∗ ^with rate of convergence

O(β

_n

)

, and write

x

_n

= x

^∗

+ O(β

_n

)

^.

Example 1 Compare the convergence behavior of

{x

ⁿ

}

^and

{y

ⁿ

}

^{, where}

x

_n

= n + 1

n

²

,

^and

y

_n

= n + 3 n

³

.

(8)

Sol. Non-linear Fun.

⁴

c

^and

α

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

^α

, ∀ n ≥ N,

α

^.

{β

ⁿ

} → 0

^and

{x

ⁿ

} → x

^∗^{. If}

∃ c > 0

and an integer

N > 0

such that

|x

ⁿ

− x

^∗

| ≤ c|β

ⁿ

|, ∀ n ≥ N,

then we say

{x

ⁿ

}

^converges ^to

x

O(β

_n

)

, and write

x

_n

= x

^∗

+ O(β

_n

)

^.

{x

ⁿ

}

^and

{y

ⁿ

}

^{, where}

x

_n

= n + 1

n

²

,

^and

y

_n

= n + 3 n

³

.

(9)

Sol. Non-linear Fun.

⁴

c

^and

α

and an integer

N > 0

^{such that}

|x

ⁿ⁺¹

− x

^∗

| ≤ c|x

ⁿ

− x

^∗

|

^α

, ∀ n ≥ N,

α

^.

{β

ⁿ

} → 0

^and

{x

ⁿ

} → x

^∗^{. If}

∃ c > 0

and an integer

N > 0

such that

|x

ⁿ

− x

^∗

| ≤ c|β

ⁿ

|, ∀ n ≥ N,

then we say

{x

ⁿ

}

^converges ^to

x

O(β

_n

)

, and write

x

_n

= x

^∗

+ O(β

_n

)

^.

{x

ⁿ

}

^and

{y

ⁿ

}

^{, where}

x

_n

= n + 1

n

²

,

^and

y

_n

= n + 3 n

³

.

(10)

Sol. Non-linear Fun.

⁵

Solution: Note that both

n→∞

lim x

_n

= 0

^and

lim

n→∞

y

_n

= 0.

Let

α

_n

=

_n¹ ^and

β

_n

=

_n¹2. Then

|x

ⁿ

− 0| = n + 1

n

²

≤ n + n

n

²

= 2

n = 2α

_n

,

|y

ⁿ

− 0| = n + 3

n

³

≤ n + 3n

n

³

= 4

n

²

= 4β

_n

.

Hence

x

_n

= 0 + O 1 n

and

y

_n

= 0 + O 1 n

²

.

This shows that

{y

ⁿ

}

converges to 0 much faster than

{x

ⁿ

}

^.

(11)

Sol. Non-linear Fun.

⁵

n→∞

lim x

_n

= 0

^and

lim

n→∞

y

_n

= 0.

Let

α

_n

=

_n¹ ^and

β

_n

=

_n¹2.

Then

|x

ⁿ

− 0| = n + 1

n

²

≤ n + n

n

²

= 2

n = 2α

_n

,

|y

ⁿ

− 0| = n + 3

n

³

≤ n + 3n

n

³

= 4

n

²

= 4β

_n

.

Hence

x

_n

= 0 + O 1 n

and

y

_n

= 0 + O 1 n

²

.

This shows that

{y

ⁿ

}

{x

ⁿ

}

^.

(12)

Sol. Non-linear Fun.

⁵

n→∞

lim x

_n

= 0

^and

lim

n→∞

y

_n

= 0.

Let

α

_n

=

_n¹ ^and

β

_n

=

_n¹2. Then

|x

ⁿ

− 0| = n + 1

n

²

≤ n + n

n

²

= 2

n = 2α

_n

,

|y

ⁿ

− 0| = n + 3

n

³

≤ n + 3n

n

³

= 4

n

²

= 4β

_n

.

Hence

x

_n

= 0 + O 1 n

and

y

_n

= 0 + O 1 n

²

.

This shows that

{y

ⁿ

}

{x

ⁿ

}

^.

(13)

Sol. Non-linear Fun.

⁵

n→∞

lim x

_n

= 0

^and

lim

n→∞

y

_n

= 0.

Let

α

_n

=

_n¹ ^and

β

_n

=

_n¹2. Then

|x

ⁿ

− 0| = n + 1

n

²

≤ n + n

n

²

= 2

n = 2α

_n

,

|y

ⁿ

− 0| = n + 3

n

³

≤ n + 3n

n

³

= 4

n

²

= 4β

_n

.

Hence

x

_n

= 0 + O 1 n

and

y

_n

= 0 + O 1 n

²

.

This shows that

{y

ⁿ

}

{x

ⁿ

}

^.

(14)

Sol. Non-linear Fun.

⁵

n→∞

lim x

_n

= 0

^and

lim

n→∞

y

_n

= 0.

Let

α

_n

=

_n¹ ^and

β

_n

=

_n¹2. Then

|x

ⁿ

− 0| = n + 1

n

²

≤ n + n

n

²

= 2

n = 2α

_n

,

|y

ⁿ

− 0| = n + 3

n

³

≤ n + 3n

n

³

= 4

n

²

= 4β

_n

.

Hence

x

_n

= 0 + O 1 n

and

y

_n

= 0 + O 1 n

²

.

This shows that

{y

ⁿ

}

{x

ⁿ

}

^.

(15)

Sol. Non-linear Fun.

⁶

2 – Bisection Method

Idea: if

f (x) ∈ C[a, b]

^and

f (a)f (b) < 0

^{, then}

∃ c ∈ (a, b)

^{such that}

f (c) = 0

^.

Algorithm 1 (Bisection Method) Given

f (x)

^{defined on}

(a, b)

, the maximal number of iterations

M

, and stop criteria

δ

^and

ε

, this algorithm tries to locate one root of

f (x)

^.

compute

u = f (a)

^,

v = f (b)

^{, and}

e = b − a

^.

if

sign(u) = sign(v)

, then stop for

k = 1, 2, . . . , M

^do

e = e/2

^,

c = a + e

^,

w = f (c)

^.

if

|e| < δ

^or

|w| < ε

, then stop if

sign(w) 6= sign(u)

^then

b = c

^,

v = w

^.

else

a = c

^,

u = w

end if end for

(16)

Sol. Non-linear Fun.

⁷

Let

{c

ⁿ

}

be the sequence of numbers produced. The algorithm should stop if one of the following conditions is satisfied.

1. the iteration number

k > M

^,

2.

|c

^k

− c

^k−1

| < δ

^{, or}

3.

|f(c

^k

)| < ε

^.

Let

[a

₀

, b

₀

], [a

₁

, b

₁

], . . .

denote the successive intervals produced by the bisection algorithm. Then

a = a

₀

≤ a

¹

≤ a

²

≤ · · · ≤ b

⁰

= b

⇒ {a

ⁿ

}

^and

{b

ⁿ

}

are bounded.

⇒ lim

n→∞

a

_n ^and

lim

n→∞

b

_n ^exist.

(17)

Sol. Non-linear Fun.

⁷

Let

{c

ⁿ

}

k > M

^,

2.

|c

^k

− c

^k−1

| < δ

^{, or}

3.

|f(c

^k

)| < ε

^.

Let

[a

₀

, b

₀

], [a

₁

, b

₁

], . . .

denote the successive intervals produced by the bisection algorithm.

Then

a = a

₀

≤ a

¹

≤ a

²

≤ · · · ≤ b

⁰

= b

⇒ {a

ⁿ

}

^and

{b

ⁿ

}

are bounded.

⇒ lim

n→∞

a

_n ^and

lim

n→∞

b

_n ^exist.

(18)

Sol. Non-linear Fun.

⁷

Let

{c

ⁿ

}

k > M

^,

2.

|c

^k

− c

^k−1

| < δ

^{, or}

3.

|f(c

^k

)| < ε

^.

Let

[a

₀

, b

₀

], [a

₁

, b

₁

], . . .

a = a

₀

≤ a

¹

≤ a

²

≤ · · · ≤ b

⁰

= b

⇒ {a

ⁿ

}

^and

{b

ⁿ

}

are bounded.

⇒ lim

n→∞

a

_n ^and

lim

n→∞

b

_n ^exist.

(19)

Sol. Non-linear Fun.

⁷

Let

{c

ⁿ

}

k > M

^,

2.

|c

^k

− c

^k−1

| < δ

^{, or}

3.

|f(c

^k

)| < ε

^.

Let

[a

₀

, b

₀

], [a

₁

, b

₁

], . . .

a = a

₀

≤ a

¹

≤ a

²

≤ · · · ≤ b

⁰

= b

⇒ {a

ⁿ

}

^and

{b

ⁿ

}

are bounded.

⇒ lim

n→∞

a

_n ^and

lim

n→∞

b

_n ^exist.

(20)

Sol. Non-linear Fun.

⁷

Let

{c

ⁿ

}

k > M

^,

2.

|c

^k

− c

^k−1

| < δ

^{, or}

3.

|f(c

^k

)| < ε

^.

Let

[a

₀

, b

₀

], [a

₁

, b

₁

], . . .

a = a

₀

≤ a

¹

≤ a

²

≤ · · · ≤ b

⁰

= b

⇒ {a

ⁿ

}

^and

{b

ⁿ

}

are bounded.

⇒ lim

n→∞

a

_n ^and

lim

n→∞

b

_n ^exist.

(21)

Sol. Non-linear Fun.

⁸

Since

b

₁

− a

¹

= 1

2 (b

₀

− a

⁰

) b

₂

− a

²

= 1

2 (b

₁

− a

¹

) = 1

4 (b

₀

− a

⁰

)

.. .

b

_n

− a

ⁿ

= 1

2

ⁿ

(b

₀

− a

⁰

)

hence

n→∞

lim b

_n

− lim

n→∞

a

_n

= lim

n→∞

(b

_n

− a

ⁿ

) = lim

n→∞

1

2

ⁿ

(b

₀

− a

⁰

) = 0.

Therefore

n→∞

lim a

_n

= lim

n→∞

b

_n

≡ z.

Since

f

is a continuous function

n→∞

lim f (a

_n

) = f ( lim

n→∞

a

_n

) = f (z)

^and

lim

n→∞

f (b

_n

) = f ( lim

n→∞

b

_n

) = f (z).

(22)

Sol. Non-linear Fun.

⁸

Since

b

₁

− a

¹

= 1

2 (b

₀

− a

⁰

) b

₂

− a

²

= 1

2 (b

₁

− a

¹

) = 1

4 (b

₀

− a

⁰

)

.. .

b

_n

− a

ⁿ

= 1

2

ⁿ

(b

₀

− a

⁰

)

hence

n→∞

lim b

_n

− lim

n→∞

a

_n

= lim

n→∞

(b

_n

− a

ⁿ

) = lim

n→∞

1

2

ⁿ

(b

₀

− a

⁰

) = 0.

Therefore

n→∞

lim a

_n

= lim

n→∞

b

_n

≡ z.

Since

f

n→∞

lim f (a

_n

) = f ( lim

n→∞

a

_n

) = f (z)

^and

lim

n→∞

f (b

_n

) = f ( lim

n→∞

b

_n

) = f (z).

(23)

Sol. Non-linear Fun.

⁸

Since

b

₁

− a

¹

= 1

2 (b

₀

− a

⁰

) b

₂

− a

²

= 1

2 (b

₁

− a

¹

) = 1

4 (b

₀

− a

⁰

)

.. .

b

_n

− a

ⁿ

= 1

2

ⁿ

(b

₀

− a

⁰

)

hence

n→∞

lim b

_n

− lim

n→∞

a

_n

= lim

n→∞

(b

_n

− a

ⁿ

) = lim

n→∞

1

2

ⁿ

(b

₀

− a

⁰

) = 0.

Therefore

n→∞

lim a

_n

= lim

n→∞

b

_n

≡ z.

Since

f

n→∞

lim f (a

_n

) = f ( lim

n→∞

a

_n

) = f (z)

^and

lim

n→∞

f (b

_n

) = f ( lim

n→∞

b

_n

) = f (z).

(24)

Sol. Non-linear Fun.

⁸

Since

b

₁

− a

¹

= 1

2 (b

₀

− a

⁰

) b

₂

− a

²

= 1

2 (b

₁

− a

¹

) = 1

4 (b

₀

− a

⁰

)

.. .

b

_n

− a

ⁿ

= 1

2

ⁿ

(b

₀

− a

⁰

)

hence

n→∞

lim b

_n

− lim

n→∞

a

_n

= lim

n→∞

(b

_n

− a

ⁿ

) = lim

n→∞

1

2

ⁿ

(b

₀

− a

⁰

) = 0.

Therefore

n→∞

lim a

_n

= lim

n→∞

b

_n

≡ z.

Since

f

n→∞

lim f (a

_n

) = f ( lim

n→∞

a

_n

) = f (z)

^and

lim

n→∞

f (b

_n

) = f ( lim

n→∞

b

_n

) = f (z).

(25)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

The limit of the sequences

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

This proves the following theorem. Department of Mathematics – NTNU Tsung-Min Hwang November 16, 2003

(26)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

(27)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

(28)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

(29)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^.

Then

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

(30)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

(31)

Sol. Non-linear Fun.

⁹

Since

f (a

_n

)f (b

_n

) ≤ 0

^.

⇒ lim

_n→∞

f (a

_n

)f (b

_n

) = f

²

(z) ≤ 0

^.

⇒ f(z) = 0

^.

⇒

{a

ⁿ

}

^and

{b

ⁿ

}

is a zero of

f

ⁱⁿ

[a, b]

^.

Let

c

_n

=

¹₂

(a

_n

+ b

_n

)

^{. Then}

|z − c

ⁿ

| =

lim

n→∞

a

_n

− 1

2 (a

_n

+ b

_n

)

=

1 2

h lim

n→∞

a

_n

− b

ⁿ

i

+ 1 2

h lim

n→∞

a

_n

− a

ⁿ

i

≤ 1

2 max

lim

n→∞

a

_n

− b

ⁿ

,

lim

n→∞

a

_n

− a

ⁿ

≤ 1

2 |b

ⁿ

− a

ⁿ

| = 1

2

ⁿ⁺¹

|b

⁰

− a

⁰

|.

This proves the following theorem.

(32)

Sol. Non-linear Fun.

¹⁰

Theorem 1 Let

{[a

ⁿ

, b

_n

]}

denote the intervals produced by the bisection algorithm. Then

n→∞

lim a

_n ^and

lim

n→∞

b

_n ^{exist, are} equal, and represent a zero of

f (x)

^{. If}

z = lim

n→∞

a

_n

= lim

n→∞

b

_n ^and

c

_n

= 1

2 (a

_n

+ b

_n

),

then

|z − c

ⁿ

| ≤ 1

2

ⁿ⁺¹

(b

₀

− a

⁰

) .

Remarks 1

{c

ⁿ

}

^converges ^to

z

^{with the} ^rate ^of

O(2

⁻ⁿ

)

^.

Example 2 If bisection method starts with interval

[50, 75]

, then how many steps should be taken to compute a root with relative error that is less than

10

⁻¹²^?

(33)

Sol. Non-linear Fun.

¹⁰

Theorem 1 Let

{[a

ⁿ

, b

_n

]}

n→∞

lim a

_n ^and

lim

n→∞

b

f (x)

^{. If}

z = lim

n→∞

a

_n

= lim

n→∞

b

_n ^and

c

_n

= 1

2 (a

_n

+ b

_n

),

then

|z − c

ⁿ

| ≤ 1

2

ⁿ⁺¹

(b

₀

− a

⁰

) .

Remarks 1

{c

ⁿ

}

^converges ^to

z

O(2

⁻ⁿ

)

^.

[50, 75]

10

⁻¹²^?

(34)

Sol. Non-linear Fun.

¹⁰

Theorem 1 Let

{[a

ⁿ

, b

_n

]}

n→∞

lim a

_n ^and

lim

n→∞

b

f (x)

^{. If}

z = lim

n→∞

a

_n

= lim

n→∞

b

_n ^and

c

_n

= 1

2 (a

_n

+ b

_n

),

then

|z − c

ⁿ

| ≤ 1

2

ⁿ⁺¹

(b

₀

− a

⁰

) .

Remarks 1

{c

ⁿ

}

^converges ^to

z

O(2

⁻ⁿ

)

^.

[50, 75]

10

⁻¹²^?

(35)

Sol. Non-linear Fun.

¹¹

Solution: Seek an

n

^{such that}

|z − c

ⁿ

|

|z| ≤ 10

⁻¹²

.

Since the bisection method starts with the interval

[50, 75]

, this implies that

z ≥ 50

^{, hence}

it is sufficient to show

|z − c

ⁿ

|

|z| ≤ |z − c

ⁿ

|

50 ≤ 10

⁻¹²

.

That is, we solve

2

⁻⁽ⁿ⁺¹⁾

(75 − 50) ≤ 50 × 10

⁻¹²

for

n

, which gives

n ≥ 38

^.

(36)

Sol. Non-linear Fun.

¹¹

Solution: Seek an

n

^{such that}

|z − c

ⁿ

|

|z| ≤ 10

⁻¹²

.

[50, 75]

, this implies that

z ≥ 50

^,

hence it is sufficient to show

|z − c

ⁿ

|

|z| ≤ |z − c

ⁿ

|

50 ≤ 10

⁻¹²

.

That is, we solve

2

⁻⁽ⁿ⁺¹⁾

(75 − 50) ≤ 50 × 10

⁻¹²

for

n

, which gives

n ≥ 38

^.

(37)

Sol. Non-linear Fun.

¹¹

Solution: Seek an

n

^{such that}

|z − c

ⁿ

|

|z| ≤ 10

⁻¹²

.

[50, 75]

, this implies that

z ≥ 50

^{, hence}

|z − c

ⁿ

|

|z| ≤ |z − c

ⁿ

|

50 ≤ 10

⁻¹²

.

That is, we solve

2

⁻⁽ⁿ⁺¹⁾

(75 − 50) ≤ 50 × 10

⁻¹²

for

n

, which gives

n ≥ 38

^.

(38)

Sol. Non-linear Fun.

¹¹

Solution: Seek an

n

^{such that}

|z − c

ⁿ

|

|z| ≤ 10

⁻¹²

.

[50, 75]

, this implies that

z ≥ 50

^{, hence}

|z − c

ⁿ

|

|z| ≤ |z − c

ⁿ

|

50 ≤ 10

⁻¹²

.

That is, we solve

2

⁻⁽ⁿ⁺¹⁾

(75 − 50) ≤ 50 × 10

⁻¹²

for

n

, which gives

n ≥ 38

^.

(39)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

⁰⁰ exists and is continuous.

If

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

is small, then by Taylor’s theorem

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

is negligible. It is reasonable to drop

O(h

²

)

terms. This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.

(40)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

⁰⁰ exists and is continuous. If

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

^{is small,}

then by Taylor’s theorem

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

O(h

²

)

terms. This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.

(41)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

O(h

²

)

terms. This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.

(42)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

is negligible.

It is reasonable to drop

O(h

²

)

terms. This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.

(43)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

O(h

²

)

^terms.

This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.

(44)

Sol. Non-linear Fun.

¹²

3 – Newton’s Method

3.1 – Derivation of Newton’s Method

Suppose that

f : R → R

^and

f ∈ C

²

[a, b]

^{, i.e.,}

f

f (x

^∗

) = 0

^and

x

^∗

= x + h

^where

h

0 = f (x

^∗

) = f (x + h) = f (x) + f

⁰

(x)h + 1

2 f

⁰⁰

(x)h

²

+ 1

3! f

⁰⁰⁰

(x)h

³

+ · · ·

= f (x) + f

⁰

(x)h + O(h

²

).

Since

h

^is ^small,

O(h

²

)

O(h

²

)

terms. This implies

f (x) + f

⁰

(x)h ≈ 0

^and

h ≈ − f (x)

f

⁰

(x) ,

^if

f

⁰

(x) 6= 0.