**Chapter 2**

**Solutions of Equations in One** **Variable**

**Hung-Yuan Fan (范洪源)**

**Department of Mathematics,**
**National Taiwan Normal University, Taiwan**

**Spring 2016**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Section 2.1**

**The Bisection Method**

**(二分法)**

### Solutions of Nonlinear Equations

**Root-Finding Problem (勘根問題)**

One of the most basic problems in numerical analysis.

* Try to find a root (or solution) p of a nonlinear equation of*
the form

*f(x) = 0,*

*given a real-valued function f, i.e. f(p) = 0.*

**The root p is also called a zero (零根) of f.**

**Note: Three numerical methods will be discussed here:**

Bisection method

*Newton’s (or Newton-Raphson) method*

*Secant and False Position (or Regula Falsi) methods*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### The Procedure of Bisection Method

*Assume that f is well-defined on the interval [a, b].*

**Set a**_{1}**= a and b****1** * = b. Find the midpoint p*1

*of [a*

_{1}

*, b*1] by

*p*

_{1}

**= a**

**1**+

**b**

_{1}

**− a****1**

**2**

= *a*

_{1}

*+ b*

_{1}

2 *.*

*If f(p*_{1}**) = 0, set p = p****1** and we are done.

*If f(p*_{1})*̸= 0, then we have*

*f(p*1)*· f(a*1*) > 0**⇒ p ∈ (p*1*, b*1**). Set a****2****= p****1****and b****2****= b****1**.
*f(p*1)*· f(a*1*) < 0**⇒ p ∈ (a*1*, p*1**). Set a****2****= a****1****and b****2****= p****1**.
Continue above process until convergence.

### Illustrative Diagram of Bisection Method

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Pseudocode of Bisection Method

*Given f∈ C[a, b] with f(a) · f(b) < 0 .*

**Algorithm 2.1: Bisection**

INPUT *endpoints a, b; tolerance TOL; max. no. of iter. N*_{0}.
OUTPUT *an approx. sol. p.*

Step 1 *Set i = 1 and FA = f(a);*

Step 2 *While i≤ N*0 **do Steps 3–6**

Step 3 *Set p = a + (b**− a)/2; FP = f(p).*

Step 4 *If FP = 0 or (b***− a)/2 < TOL then OUTPUT(p); STOP.**

Step 5 *Set i = i + 1.*

Step 6 *If FP**· FA > 0 then set a = p and FA = FP.*

*Else set b = p. (FA is unchanged)*

Step 7 *OUTPUT(‘Method failed after N*_{0} **iterations’) and STOP.**

**Stopping Criteria (停止準則)**

*In Step 4, other stopping criteria can be used. Let ϵ > 0 be a given*
*tolerance and p*_{1}*, p*2*, . . . , p**N* be generated by Bisection method.

(1) *|p**N**− p**N**−1**| < ϵ,*

(2) ^{|p}^{N}^{−p}_{|p}^{N}^{−1}^{|}

*N**|* *< ϵ with p*_{N}*̸= 0,*
(3) *|f(p**N*)*| < ϵ.*

**Note: The stopping criterion (2) is preferred in practice.**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### An Example for Bisection Method

**Example 1, p. 50**

(1) Show that the equation

*f(x) = x*^{3}*+ 4x*^{2}*− 10 = 0*
*has exactly one root in [1, 2].*

(2) Use Bisection method to determine an approx. root which is
*accurate to at least within 10** ^{−4}*.

**The root is p = 1.365230013 correct to 9 decimal places.**

**Solution**

(1) *By IVT with f(1)f(2) = (−5)(14) < 0, ∃ p ∈ (1, 2) s.t.*

*f(p) = 0. Since f*^{′}*(x) = 3x*^{2}*+ 8x > 0 for x∈ (1, 2), the root*
*must be unique in [1, 2].*

(2) **After 13 iterations, since** *|a*14*| < |p|, we have*

*|p − p*13*|*

*|p|* *≤* *|b*14*− a*14*|*

*|a*14*|* **≤ 9.0 × 10**^{−5}*.*
Note that

*|f(p*9)*| < |f(p*13)*|*
in the Table 2.1.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Numerical Results for Example 1

### Convergence of the Bisection Method

**Thm 2.1 (二分法的收斂定理)**

*Suppose that f∈ C[a, b] with f(a) · f(b) < 0. The Bisection method*
generates a sequence*{p**n**}*^{∞}*n=1* *converging to a root p of f with*

*|p − p**n**| ≤* *b− a*

2^{n}*∀ n ≥ 1.*

**The rate of convergence is O(**_{2}^{1}**n**).

**pf: For each n**

**pf: For each n**

*≥ 1, p ∈ (a*

*n*

*, b*

*n*) and

*b*

_{n}*− a*

*n*=

*b− a*

**2**

^{n}*by induction.*

^{−1}Hence, we hve

*|p − p**n**| ≤* *b*_{n}*− a**n*

2 = *b− a*
2^{n}*.*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Is the error bound tight?

**Remark**

Applying Thm 2.1 to Example 1, we see that

*|p − p*9*| ≤* 2*− 1*

2^{9} **≈ 2 × 10**^{−3}*,*

but the actual absolute value is*|p − p*9**| ≈ 4.4 × 10**^{−6}*. In this case,*
**the error bound in Thm 2.1 is much larger than the actual error.**

**Example 2, p. 52**

*As in Example 1, use Thm 2.1 to estimate the smallest number N*
of iterations so that*|p − p**N**| < 10** ^{−3}*.

**Sol: Applying Thm 2.1, it follows that**

*|p − p**N**| ≤* 2*− 1*

2^{N}*< 10*^{−3}*⇐⇒ 2*^{−N}*< 10*^{−3}*,*
or, equivalently, (*−N) log*_{10}*2 <−3 ⇐⇒ N >* _{log}^{3}

102 **≈ 9.96. So,**

**10 iterations will ensure the required accuracy. But, in fact, we**

know that
*|p − p*9**| ≈ 4.4 × 10**^{−6}*.*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Useful Suggestions for the Bisection Method

**In Practical Computation...**

To avoid the round-off errors in the computations, use
*p*_{n}*= a**n*+*b*_{n}*− a**n*

2 instead of *p** _{n}*=

*a*

_{n}*+ b*

*n*

2 *.*

*To avoid the overflow or underflow of f(p** _{n}*)

*· f(a*

*n*), use

*sign(f(p*

*))*

_{n}*· sign(f(a*

*n*

*)) < 0*instead of

*f(p*

*)*

_{n}*· f(a*

*n*

*) < 0.*

**Note: The sign function is defined by**

*sign(x) =*

1 *if x > 0,*
0 *if x = 0,*

*−1 if x < 0.*

**Section 2.2**

**Fixed-Point Iteration**

**(固定點迭代)**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Def 2.2**

*The number p is called a fixed point (固定點) of a real-valued*
*function g if g(p) = p.*

**Note: A root-finding problem of the form**

*f(p) = 0,*

**where p is a root of f, can be transformed to a fixed-point form***p = g(p),*

*for some suitable function g obtained by algebraic transposition.*

*(函數 g(x) 經由原函數 f(x) 代數移項可得)*

### Existence & Uniqueness of Fixed Points

**Thm 2.3 (固定點的存在性與唯一性)**

(i) If*g∈ C[a, b]*and*g([a, b])⊆ [a, b] , then g has at least one*

*fixed point in [a, b].*

(ii) If, in addition, *g*^{′}*(x)* *exists on (a, b) and* *∃ 0 < k < 1 s.t.*

*|g*^{′}*(x)| ≤ k ∀ x ∈ (a, b),*
**then there is exactly one fixed point in [a, b].**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Illustrative Diagram for Fixed Points

*Geometrically, a fixed point p∈ [a, b] is just the point where the*
curves*y = g(x)* and*y = x*intersect.

**Proof of Thm 2.3**

(i) *If g(a) = a or g(b) = b, we are done. If not, then g(a) > a*
*and g(b) < b, since g([a, b])⊆ [a, b]. Note that the function*
*h(x) = g(x)− x ∈ C[a, b] and*

*h(a) = g(a)− a > 0, h(b) = g(b) − b < 0.*

By IVT, *∃ p ∈ (a, b) s.t. h(p) = 0 or g(p) = p.*

(ii) Suppose that *∃ p ̸= q ∈ [a, b] s.t. g(p) = p and g(q) = q . By*
MVT, *∃ ξ between p and q s.t.*

*|p − q| = |g(p) − g(q)| = |g*^{′}*(ξ)||p − q|*

**≤ k|p − q|< |p − q|,**

**which is a contradiction! Hence, g must have a unique fixed***point in [a, b].*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Example 3: Condition (ii) Is NOT Satisfied (1/2)

**Example 3, p. 59**

*Although the sufficient conditions are NOT satisfied for g(x) = 3*^{−x}**on the interval [0, 1], there does exist a unique fixed point of g in***[0, 1].*

**Sol: Since g**

**Sol: Since g**

^{′}*(x) =−3*

^{−x}*ln 3 < 0*

*∀ x ∈ [0, 1], g is strictly*

*decreasing on [0, 1] and hence*

1

3 *= g(1)≤ g(x) ≤ g(0) = 1 ∀ x ∈ [0, 1],*
*i.e. g∈ C[0, 1] and g([0, 1]) ⊆ [0, 1].*

### Example 3: Condition (ii) Is NOT Satisfied (2/2)

But also note that

*g** ^{′}*(0) =

*− ln 3 ≈ −1.0986,*

thus*|g*^{′}*(x)| ≮ 1 on (0, 1) and condition (ii) of Thm 2.3 is not*
*satisfied. Because g is strictly deceasing on [0, 1], its graph must*
**intersect the graph of y = x at exactly one fixed point p**∈ (0, 1).

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Fixed-Point Iteration (固定點迭代)

**Functional (or Fixed-Point) Iteration**

*Assume that g∈ C[a, b] and g([a, b]) ⊆ [a, b]. The fixed-point*
iteration generates a sequence*{p**n**}*^{∞}_{n=1}*, with p*_{0} *∈ [a, b], defined by*

*p*_{n}*= g(p**n**−1*) *∀ n ≥ 1.*

**This method is also called the functional iteration. (泛函迭代)**

### Illustrative Diagrams

*Starting wirh p*_{0}*∈ [a, b], we obtain a sequence of points*

*(p*_{0}*, p*_{1})*→ (p*1*, p*_{1})*→ (p*1*, p*_{2})*→ (p*2*, p*_{2})*→ (p*2*, p*_{3})*→ · · ·(p, p),*
where*p = g(p).*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Pseudocode of Functional Iteration

*To find a sol. p to x = g(x) given an initial approx. p*_{0}.

**Algorithm 2.2: Fixed-Point Iteration**

INPUT *initial approx. p*_{0}*; tolerance TOL; max. no. of iter. N*_{0}.
OUTPUT *approx. sol. p to x = g(x).*

Step 1 *Set i = 1.*

Step 2 *While i≤ N*0 **do Steps 3–6**
Step 3 *Set p = g(p*0).

Step 4 If*|p − p*0**| < TOL then OUTPUT(p); STOP.**

Step 5 *Set i = i + 1.*

Step 6 *Set p*0*= p. (Update p*0)

Step 7 *OUTPUT(‘Method failed after N*_{0} **iterations’); STOP.**

### An Illustrative Example

**5 Possible Fixed-Point Forms**

The root-finding problem
*f(x) = x*^{3}*+ 4x*^{2}*− 10 = 0*

can be transformed to the following 5 fixed-point forms:

*(a) x = g*_{1}*(x) = x− x*^{3}*− 4x*^{2}+ 10 *(b) x = g*_{2}*(x) = (*10

*x* *− 4x)*^{1/2}
*(c) x = g*_{3}*(x) =* 1

2(10*− x*^{3})^{1/2} *(d) x = g*_{4}*(x) = (* 10
*4 + x*)^{1/2}
*(e) x = g*_{5}*(x) = x−* *x*^{3}*+ 4x*^{2}*− 10*

*3x*^{2}*+ 8x*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

*Numerical Results with p*

0 *= 1.5*

**Some Questions**

Under what conditions does the fixed-point iteration (FPI)
*p*_{n}*= g(p**n**−1**),* *n = 1, 2, . . .*

converge*for any p*_{0} *∈ [a, b]*?

What is the error bound for the FPI?

In addition, what is the rate of convergence?

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Convergence of Functional Iteration

**Thm 2.4 (Fixed-Point Thm)**

*Suppose that g∈ C[a, b] and g([a, b]) ⊆ [a, b]. If g*^{′}*(x) exists on*
*(a, b) and∃ k ∈ (0, 1) s.t.*

*|g*^{′}*(x)| ≤ k ∀ x ∈ (a, b),*

then*for any p*_{0}*∈ [a, b]*, the sequence *{p**n**}*^{∞}*n=1* defined by
*p*_{n}*= g(p**n**−1*) *∀ n ≥ 1,*

**converges to the unique fixed point p**∈ [a, b] of g.

### Proof of Thm 2.4

Thm 2.3 ensure that *∃!p∈ [a, b] s.t. g(p) = p.*

*For each n≥ 1, it follows from MVT that ∃ ξ**n* *between p*_{n}_{−1}*and p s.t.*

*|p**n**− p| = |g(p**n**−1*)*− g(p)| = |g*^{′}*(ξ**n*)*||p**n**−1**− p| ≤ k|p**n**−1**− p|.*

By induction =*⇒|p**n**− p| ≤ k*^{n}*|p*0*− p| for n ≥ 0.*

*Since 0 < k < 1, we see that*

*n*lim*→∞**|p**n**− p| = 0 ⇐⇒ lim*

*n**→∞**p*_{n}*= p*
by the Sandwich Thm.

**[Q]: What is the order of convergence for the FPI?**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Error Bounds for Fixed-Point Iteration

**Cor 2.5 (固定點迭代的誤差上界)**

*If g satisfies the hypotheses of Thm 2.4, then we have*
(1) *|p**n**− p| ≤ k** ^{n}*max

*{p*0

*− a, b − p*0

*} ∀ n ≥ 0,*(2)

*|p*

*n*

*− p| ≤ k*1

*− k |*

^{n}*p*

_{1}

*− p*0

*| ∀ n ≥ 1.*

**pf: Inequality (1) followss immediately from the proof of Thm 2.4.**

*For m > n≥ 1, by MVT inductively, we obtain*

*|p**m**− p**n**| ≤ |p**m**− p**m**−1**| + |p**m**−1**− p**m**−2**| + · · · + |p**n+1**− p**n**|*

*≤ k*^{m}^{−1}*|p*1*− p*0*| + k*^{m}^{−2}*|p*1*− p*0*| + · · · + k*^{n}*|p*1*− p*0*|*

*= k*^{n}*(1 + k + k*^{2}+*· · · + k*^{m}* ^{−n−1}*)

*· |p*1

*− p*0

*|.*

*Hence, by taking m→ ∞, we have*

*|p − p**n**| = lim*

*m**→∞**|p**m**− p**n**| ≤ k** ^{n}*(∑

^{∞}*i=0*

*k** ^{i}*)

*|p*1*− p*0*| =* *k*^{n}

1*− k|p*1*− p*0*|.*

### Rate of Conv. for Functional Iter.

**Remarks**

The rate of convergence for the fixed-point iteration depends
on*k** ^{n}* or

_{1}

^{k}

_{−k}*.*

^{n}*The smaller the value of k, the faster the convergence.*

**The convergence would be very slow if k**≈ 1.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### The Illustrative Example Revisited

**5 Possible Fixed-Point Forms**

The root-finding problem
*f(x) = x*^{3}*+ 4x*^{2}*− 10 = 0*

can be transformed to the following 5 fixed-point forms:

*(a) x = g*_{1}*(x) = x− x*^{3}*− 4x*^{2}+ 10 *(b) x = g*_{2}*(x) = (*10

*x* *− 4x)*^{1/2}
*(c) x = g*_{3}*(x) =* 1

2(10*− x*^{3})^{1/2} *(d) x = g*_{4}*(x) = (* 10
*4 + x*)^{1/2}
*(e) x = g*_{5}*(x) = x−* *x*^{3}*+ 4x*^{2}*− 10*

*3x*^{2}*+ 8x*

### Illustration

(a) *g*_{1}*([1, 2])* [1, 2] and |g*^{′}_{1}*(x)| > 1 for x ∈ [1, 2].*

(b) *g*_{2}*([1, 2])* [1, 2] and |g** ^{′}*2

*(x)| ≮ 1 for any interval containing*

*p*

**≈ 1.36523, since |g**

^{′}

_{2}**(p)**

**| ≈ 3.4.**(c) *Since p*_{0} *= 1.5, 1 < 1.28≈ g*3*(1.5)≤ g*3*(x)≤ g*3*(1) = 1.5*
**and hence g**_{3}**([1, 1.5])****⊆ [1, 1.5]. We also note that g**^{′}_{3}
satisfies*|g** ^{′}*3

*(x)| ≤ |g*

*3*

^{′}*(1.5)*

**| ≈ 0.66 for x ∈ [1, 1.5].**(d) *g*_{4}*([1, 2])⊆ [1, 2] and the derivative g** ^{′}*4 satisfies

*|g*^{′}_{4}*(x)| =**√* *−5*
*10(4 + x)*^{3/2}

* <* 5

*√*10(5)^{3/2} **≈ 0.1414.**

(e) **It is Newton’s method satisfying g**^{′}_{5}**(p) = 0 theoretically!**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Section 2.3**

**Newton’s Method and Its Extensions**

**(牛頓法及其推廣)**

### Derivation of Newton’s Method

Suppose that *f(p) = 0,f*^{′}*(p)̸= 0* and*f∈ C*^{2}*[a, b].*

Given an initial approximation *p*_{0} *∈ [a, b]*with *f*^{′}*(p*0)*̸= 0* s.t.

*|p − p*0* | is sufficiently small*.

By Taylor’s Thm, *∃ ξ(p) between p and p*0 s.t.

*0 = f(p) = f(p*0*) + f*^{′}*(p*0*)(p− p*0) +*f*^{′′}*(ξ(p))*

2 *(p− p*0)^{2}*.*
Since *|p − p*0*| is sufficiently small*, it follows that

0*≈ f(p*0*) + f*^{′}*(p*_{0}*)(p− p*0)*⇐⇒ p ≈ p*0*−* *f(p*_{0})
*f*^{′}*(p*_{0})*.*
This suggests the procedure of Newton’s method:

*p*_{n}*= p*_{n}_{−1}*−* *f(p*_{n}* _{−1}*)

*f*^{′}*(p**n**−1*) *∀ n ≥ 1.*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Newton’s Method v.s. Functional Iteration

**Observations**

*Let g be a real-valued function defined by*
*g(x) = x−* *f(x)*

*f*^{′}*(x),* *x∈ [a, b],*

Newton’s method can be viewed as a fixed-point iteration
*p*_{n}*= g(p**n**−1*) **∀ n ≥ 1, where |p****0****− p| is sufficiently small.**

**If f(p) = 0, g(p) = p, i.e., p is a fixed-point of g.**

*g∈ C[a, b]*and its first derivative is given by
*g*^{′}*(x) =* *f(x)f*^{′′}*(x)*

*[f*^{′}*(x)]*^{2} *,* *x∈ [a, b].*

*If f(p) = 0, then* *g*^{′}*(p) = 0* follows immediately.

**Further Questions**

*Under what conditions does Newton’s method converge to p?*

What is the error bond for Newton’s method?

**How to choose a good initial guess p**_{0}?

What is the rate of convergence for Newton’s method?

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Pseudocode of Newton’s Method

*To find a sol. to f(x) = 0 given an initial approx. p*_{0}.

**Algorithm 2.3: Newton’s Method**

INPUT *initial approx. p*_{0}*; tolerance TOL; max. no. of iter. N*_{0}.
OUTPUT *approx. sol. p to f(x) = 0.*

Step 1 *Set i = 1.*

Step 2 *While i≤ N*0 **do Steps 3–6**
Step 3 Set*p = p*0*− f(p*0*)/f*^{′}*(p*0).

Step 4 If*|p − p*0**| < TOL then OUTPUT(p); STOP.**

Step 5 *Set i = i + 1.*

Step 6 *Set p*0*= p. (Update p*0)

Step 7 *OUTPUT(‘Method failed after N*_{0} **iterations’); STOP.**

**Example 1, p. 69**

Use (a) fixed-point iteration and (b) Newton’s method to find an
*approximate root p of the nonlinear equation*

*f(x) = cos x− x = 0*

with initial guess*p*_{0} = ^{π}_{4}*. The root is p ≈ 0.739085133215161.*

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Solution (1/3)

(a) *Consider the fixed-point form x = g(x), where*
*g(x) = cos(x)* *∀ x ∈ [0,π*

2*].*

Then it is easily seen that

**1** *g**∈ C[0,*^{π}_{2}],

**2** *g([0,*^{π}_{2}])*⊆ [0, 1] ⊆ [0,*^{π}_{2}],

**3** *|g*^{′}*(x)**| = | − sin(x)| < 1 ∀ x ∈ (0,*^{π}_{2}).

From Thm 2.4 =*⇒ the fixed-point iteration*
*p*_{n}*= g(p**n**−1**) = cos(p**n**−1*) *∀ n ≥ 1*

**must converge to the unique fixed point p**∈ (0,^{π}_{2}*) of g*for
*any initial p*_{0} *∈ [0,*^{π}_{2}]!

### Solution (2/3)

Applying the FPI with an initial guess*p*_{0} = ^{π}_{4}, we obtain the
following numerical results.

*The root is p ≈ 0.739085133215161.*

**Note that only 2 significant digits!**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Solution (3/3)

(b) *For the same initial approx. p*_{0}*= π/4, applying Newton’s*
method

*p*_{n}*= p*_{n}_{−1}*−cos(p*_{n}* _{−1}*)

*− p*

*n*

*−1*

*− sin(p**n**−1*)*− 1* *∀ n ≥ 1,*
we obtain the following numerical results

*The actual root is p ≈ 0.739085133215161.*

### Convergence Thm for Newton’s Method

**Thm 2.6 (牛頓法的收斂定理)**

*Let f∈ C*^{2}*[a, b] and p∈ (a, b). If* *f(p) = 0*and*f*^{′}*(p)̸= 0*, then

*∃δ > 0*s.t. Newton’s method generates a sequence *{p**n**}*^{∞}* _{n=1}*
defined by

*p*_{n}*= p**n**−1**−* *f(p*_{n}* _{−1}*)

*f*^{′}*(p**n**−1*) *∀ n ≥ 1*
*converging to pfor any p*_{0} *∈ [p − δ, p + δ]*.

**Note: The local convergence of Newton’s method is guaranteed**

in Thm 2.6, but the order of convergence is NOT discussed here!
. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Sketch of Proof (1/2)**

Since *f*^{′}*(p)̸= 0*,*∃ δ*1*> 0 s.t.*

*f*^{′}*(x)̸= 0 ∀ x ∈ (p − δ*1*, p + δ*1*),*
and hence

*g(x) = x−* *f(x)*
*f*^{′}*(x)*
*is well-defined on (p− δ*1*, p + δ*_{1}).

Moreover, since its derivative is given by
*g*^{′}*(x) =* *f(x)f*^{′′}*(x)*

*[f*^{′}*(x)]*^{2} *∀ x ∈ (p − δ*1*, p + δ*_{1}*),*
it follows that *g∈ C*^{1}*(p− δ*1*, p + δ*_{1}) *because f∈ C*^{2}*[a, b].*

*Note that f(p) = 0 =⇒g(p) = p*and*g*^{′}*(p) = 0.*

**Sketch of Proof (2/2)**

*Because g*^{′}*is conti. at p, for any k∈ (0, 1), ∃* *0 < δ* *< δ*_{1} s.t.

*|g*^{′}*(x)| < k ∀ x ∈ [p − δ, p + δ].*

*For x∈ [p − δ, p + δ], from MVT ⇒ ∃ ξ between x and p s.t.*

*|g(x) − p| = |g(x) − g(p)| = |g*^{′}*(ξ)| |x − p| < δ.*

Hence, *g([p− δ, p + δ]) ⊆ [p − δ, p + δ]*.

From Thm 2.4 =*⇒ the seq. generated by Newton’s method*
*p*_{n}*= g(p**n**−1*) *∀ n ≥ 1*

*converges to p* *for any p*_{0}*∈ [p − δ, p + δ]*.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Questions**

*How to guess a good initial approximation p*_{0}?
*How to estimate δ > 0 derived in Thm 2.6?*

What is the order of convergence for Newton’s method?

How to modify Newton’s method if *f*^{′}*(x) is difficult to be*
evaluated in practice? Use Secant Method!

### Derivation of Secant Method (割線法)

In many applications, it is often difficult to evaluate the
*derivative of f.*

*Since f*^{′}*(p**n**−1*) = lim

*x**→p**n−1*

*f(x)**−f(p**n−1*)

*x**−p*^{n}*−1* , we have
*f*^{′}*(p**n**−1*)*≈* *f(p*_{n}* _{−2}*)

*− f(p*

*n*

*−1*)

*p*_{n}_{−2}*− p**n**−1* = *f(p*_{n}* _{−1}*)

*− f(p*

*n*

*−2*)

*p*

_{n}

_{−1}*− p*

*n*

*−2*

*for any n≥ 2.*

With above spprox. for the derivative, Neton’s method is rewritten as

*p*_{n}*= p**n**−1**−f(p*_{n}* _{−1}*)(p

*n*

*−1*

*− p*

*n*

*−2*)

*f(p*_{n}* _{−1}*)

*− f(p*

*n*

*−2*)

*∀ n ≥ 2.*

**This is called the Secant method with initial approximations**
*p* *and p* .

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Illustrative Diagram for Secant Method

**Key Steps of Secant Method**

*Given two initial p*_{0} *and p*_{1} *with q*_{0}*← f(p*0*) and q*1*← f(p*1), the
**followings are performed repeatedly in the Secant method:**

**1** Compute the new approximation

*p← p*1*−q*_{1}*(p*_{1}*− p*0)
*q*_{1}*− q*0

;

**2** Update*p*_{0}*← p*1 and *q*_{0} *← q*1;*p*_{1}*← p* and*q*_{1}*← f(p)*.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Pseudocode of Secant Method

*To find a sol. to f(x) = 0 given initial approx. p*_{0} *and p*_{1}.

**Algorithm 2.4: Secant Method**

INPUT *initial approx. p*_{0}*, p*1*; tolerance TOL; max. no. of iter. N*_{0}.
OUTPUT *approx. sol. p to f(x) = 0.*

Step 1 *Set i = 2; q*_{0} *= f(p*_{0}*); q*_{1} *= f(p*_{1}).

Step 2 *While i≤ N*0 **do Steps 3–6**

Step 3 Set*p = p*1*− q*1*(p*1*− p*0*)/(q*1*− q*0).

Step 4 If*|p − p*1**| < TOL then OUTPUT(p); STOP.**

Step 5 *Set i = i + 1.*

Step 6 Set*p*0*= p*_{1};*q*0*= q*_{1};
*p*1*= p*;*q*1*= f(p).*

Step 7. *OUTPUT(‘Method failed after N*_{0} **iterations’); STOP.**

**Example 2, p. 72**

Use the Secant method to find a sol. to
*f(x) = cos x− x = 0*

with initial approx.*p*_{0} *= 0.5* and*p*_{1} *= π/4. Compare the results*
with those of Newton’s method obtained in Example 1.

**Sol: Applying the Secant method**

*p*_{n}*= p*_{n}_{−1}*−* *(cos p**n**−1**− p**n**−1**)(p**n**−1**− p**n**−2*)

*(cos p**n**−1**− p**n**−1*)*− (cos p**n**−2**− p**n**−2*) *∀ n ≥ 2,*
we see that its approximation*p*_{5} **is accurate to 10 significant**
digits, whereas Newton’s method produced the same accuracy
**after 3 iterations.**

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Numerical Results for Example 2

**The Secant method is much faster than fixed-point iteration,**

**but slower than Newton’s method.**

### Method of False Position (錯位法)

*The method of False Position is also called Regula Falsi*
method. **The root is always bracketed between successive**
**approximations.**

*Firstly, find p*_{2} **using the Secant method . How to determine**
*the next approx. p*_{3}?

*If f(p*2)*· f(p*1**) < 0 (or sign(f(p****2**))**· sign(f(p****1*** )) < 0), then p*3

**is the x-intercept of the line joining (p****1****, f(p****1**)) and
**(p****2****, f(p****2**)).

*If not, p*3 **is the x-intercept of the line joining (p****0****, f(p****0**)) and
**(p****2****, f(p****2****)), and then interchange the indices on p****0** **and p****1**.

Continue above procedure until convergence.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Secant Method v.s. Method of False Position

**Key Steps of False Position Method**

*Given two initial p*_{0} *and p*_{1} *with q*_{0}*← f(p*0*) and q*_{1}*← f(p*1), the
**followings are performed repeatedly in the False Position method:**

**1** Compute the new approximation

*p← p*1*−q*_{1}*(p*1*− p*0)
*q*_{1}*− q*0

;

**2** *Compute q← f(p);*

**3** *If q· q*1 *< 0, updatep*_{0} *← p*1 and *q*_{0} *← q*1;

**4** Update*p*_{1}*← p* and *q*_{1} *← q*.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Pseudocode for Method of False Position

*To find a sol. to f(x) = 0 given initial approx. p*_{0} *and p*_{1}.

**Algorithm 2.5: Method of False Position**

INPUT *initial approx. p*_{0}*, p*1*; tolerance TOL; max. no. of iter. N*_{0}.
OUTPUT *approx. sol. p to f(x) = 0.*

Step 1 *Set i = 2; q*_{0} *= f(p*_{0}*); q*_{1} *= f(p*_{1}).

Step 2 *While i≤ N*0 **do Steps 3–7**

Step 3 Set*p = p*1*− q*1*(p*1*− p*0*)/(q*1*− q*0).

Step 4 If*|p − p*1**| < TOL then OUTPUT(p); STOP.**

Step 5 *Set i = i + 1; q = f(p).*

Step 6 *If q**· q*1*< 0 then**p*0*= p*_{1};*q*0*= q*_{1}.
Step 7 Set*p*1*= p;* *q*1*= q.*

Step 8 *OUTPUT(‘Method failed after N*_{0} **iterations’); STOP.**

**Example 3, p. 74**

Use the method of False Position to find a sol. to
*f(x) = cos x− x = 0*

*with p*_{0} *= 0.5 and p*_{1} *= π/4. Compare the results with those*
obtained by Newton’s method and Secant method.

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

**Section 2.4**

**Error Analysis for Iterative Methods**

### Order of Convergence (收斂階數)

**Def 2.7 (收斂階數的定義)**

A sequence*{p**n**}*^{∞}_{n=0}*converges to p of order α, with asymptotic*
*error constant λ if∃ α ≥ 1 and λ ≥ 0 with*

*n*lim*→∞*

*|p**n+1**− p|*

*|p**n**− p|*^{α}*= λ.*

(i) *α = 1 and 0 < λ < 1 =⇒ {p**n**}*^{∞}*n=0* **is linearly convergent.**

(ii) *α = 1 and λ = 0 =⇒ {p**n**}*^{∞}*n=0* **is superlinearly convergent.**

(iii) *α = 2 =⇒ {p**n**}*^{∞}_{n=0}**is quadratically convergent.**

**Note: The higher-order convergence is always expected in**

practical computation!
. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Linear Convergence of Functional Iteration

**Thm 2.8 (固定點迭代的線性收斂性)**

*Suppose that g∈ C[a, b] and g([a, b]) ⊆ [a, b]. Ifg*^{′}*∈ C(a, b),*

*∃ k ∈ (0, 1) s.t. |g*^{′}*(x)| ≤ k ∀ x ∈ (a, b) and g*^{′}*(p)̸= 0*, thenfor
*any p*_{0} *∈ [a, b]*, the sequence

*p*_{n}*= g(p*_{n}* _{−1}*)

*∀ n ≥ 1*

**converges only linearly to the unique fixed point p**∈ [a, b].

**Proof of Thm 2.8**

*Thm 2.4 (Fixed-Point Thm) ensures that the sequence p*_{n}*converges to the unique fixed point p∈ [a, b].*

*For each n≥ 1, by MVT =⇒ ∃ ξ**n* *between p*_{n}*and p s.t.*

*|p**n+1**− p| = |g(p**n*)*− g(p)| = |g*^{′}*(ξ**n*)*||p**n**− p|.*

Since lim

*n**→∞**p*_{n}*= p, ξ**n**→ p as n → ∞. Thus,*

*n*lim*→∞*

*|p**n+1**− p|*

*|p**n**− p|* = lim

*n**→∞**|g*^{′}*(ξ**n*)| = |g^{′}*(p)| > 0*

*because g*^{′}*∈ C(a, b), i.e.,the sequence p*_{n}*converges to p only*
linearly!

. . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . .

. .

. . . . .

### Quadratic Convergence of Functional Iteration

**Thm 2.9 (固定點迭代的二次收斂性)**

If*g(p) = p,g*^{′}*(p) = 0*and*∃ open interval I containing p where*
*g*^{′′}*∈ C(I) and |g*^{′′}*(x)| < M ∀ x ∈ I,*

then*∃ δ > 0* s.t. the sequence defined by
*p*_{n}*= g(p**n**−1*) *∀ n ≥ 1*

**converges at least quadratically to p**for any p_{0} *∈ [p − δ, p + δ]*.
Moreover, we have

*|p**n+1**− p| <* *M*

2*|p**n**− p|*^{2}*for sufficiently large values of n.*