National Sun Yat-sen University Institutional Repository:Item 987654321/39888

行政院國家科學委員會專題研究計畫 成果報告

修正近點算法之研究

中 華 民 國 95 年 9 月 23 日

On Modified Proximal Point Algorithms

Jen-Chih Yao

NSC 94-2213-E-110-035

94/ 08 / 01 – 95/ 07 / 31

1. Abstract In this project, we aim to study the problem of find-ing an element x ∈ H such that 0 ∈ T (x) where H is a real Hilbert space and T is a maximal mono-tone set-valued operator from H into itself. Such problem is called inclusion problem and is very im-portant in optimization and re-lated fields. For example, if T is the subdifferential ∂f of a lower semicontinuous convex functional f : H → (−∞, ∞], then T is a maximal monotone set-valued op-erator and the inclusion problem 0 ∈ ∂f (x) is reduced to the follow-ing optimization problem:

f (x) = Min{f (z) : z ∈ H}. We plan to devise some modified proximal point algorithms for find-ing approximate solutions of the above inclusion problem. New ac-curacy criteria will be imposed.

Weak and strong convergence re-sults of the suggested algorithms will be established.

Key words : Inclusion problem, optimization, maximal monotone operator, subdifferential, proximal point algorithm.

2. Introduction

Let H be a real Hilbert space with inner product h·, ·i and norm k · k, respectively. Let T : H −→ 2H be a maximal monotone oper-ator. The problem of finding an element x ∈ H such that 0 ∈ T (x) is very important in the area of op-timization and related fields. For example, if T is the subdifferential ∂f of a proper lower semicontin-uous convex functional f : H → (−∞, ∞], then T is a maximal monotone operator and the inclu-sion 0 ∈ ∂f (x) is reduced to the

following optimization problem: f (x) = min{f (z) : z ∈ H}. One of the most efficient and en-forceable methods for solving 0 ∈ T (x) is the proximal point algo-rithm which, staring with any vec-tor x0 ∈ H, iteratively updates

xn+1 conforming to the following

recursion:

xn ∈ xn+1+ cnT (xn+1) (1)

where {cn}∞n=0 ⊂ [c, ∞) , c > 0, is

a sequence of scalars. However, as pointed out in [13], the ideal form of the method is often impractical, since in many cases solving prob-lem (1) exactly is either impossi-ble or as difficult as solving the original problem 0 ∈ T (x). On the other hand, there seems to be little justification of the effort re-quired to solve the problem accu-rately when the iterate is far away from the solution point. In [20], Rockafellar gave an inexact variant of the method:

xn+ en+1 ∈ xn+1+ cnT (xn+1) (2)

where {en+1} is regarded as an

error sequence. This method is called an inexact proximal point algorithm. Rockafellar [20] proved that if en → 0 quickly enough

such that P∞

n=1kenk < ∞, then

xn→ z ∈ Rn with 0 ∈ T (z).

Because of its relaxed accuracy requirement, the inexact proximal point algorithm is more practical than the exact one. Thus it has been studied widely and various forms of the method have been de-veloped; see, e.g., [3,4,8,10-12,17-19, 25]. In most of these papers, the condition that the error term being summable is essential for the convergence of the method. In [20] and some sequel papers (e.g., [5]) the accuracy criterion is

ken+1k ≤ ηnkxn+1− xnk (3)

with P∞

n=0ηn< ∞.

Recently, Eckstein [13] extended the method to Bregman-function-based inexact proximal methods and proved that the sequence {xn}

generated by the algorithm con-verges to a root of T under the conditions ∞ X n=1 kenk < ∞, ∞ X n=1 hen, xni < ∞ (4) (see Eqs. (18) and (19) in [13]). Condition (4) is an assumption on the whole generated sequence {xn} and the error term sequence

{en}, and thus seems to be slightly

stronger, but it can be checked and enforced in practice more eas-ily than those that existed earlier. On the other hand, as in He [15],

Han and He [14] gave another in-exact criterion

ken+1k ≤ ηnkxn+1− xnk (5)

with P∞

n=0ηn2 < ∞ to recursion

(2) for solving the equation 0 ∈ T (x) in Rn _{and studied the}

re-sulting convergence properties. It is clear that the accruacy crite-rion (5) is weaker than the one in [20] (see(3)). It is remarkable that da Silva e Silva et al. [9] and Solodov and Svaiter [21-23] re-cently proposed some new accu-racy criteria for proximal point al-gorithms. Their criteria, rather than requiring inequality (5), re-quire only sup_n≥0ηn < 1.

How-ever, in [21-23], this comes at the cost of adding an additional pro-jection or ”extragradient” step to the algorithm, and the applicable portion of [9] applies only to con-vex minimization.

Let H be a real Hilbert space and T be a maximal monotone operator on H. Throughout this project, we assume that the equa-tion 0 ∈ T (x) has a soluequa-tion and let S be the solution set:

S = {x ∈ H : 0 ∈ T (x)} = T−1(0). Then S is a nonempty closed con-vex subset of H and thus the pro-jection Ps from H onto S is well

defined. Motivated and inspired

by Xu [26], we proposed modi-fied approximate proximal point algorithms for finding approximate solutions of zeros of a maximal monotone operator in real Hilbert spaces. We introduce new ac-curacy criteria for these modi-fied approximate proximal point algorithms. Under the suggested enforceable accuracy restrictions which are easy to verify, the con-vergence results of these modified approximate proximal point algo-rithms are established. In partic-ular, results in this paper improve and extend corresponding results in [14] which were in finite dimen-sional space setting.

3. Results and Discussion We first introduced the following two algorithms.

Algorithm 3.1 (Relaxed proxi-mal point algorithm).

(i) x0 ∈ H is chosen arbitrarily.

(ii) Choose a regularization pa-rameter cn > 0 with error en+1 ∈

H and compute ˜

xn+1 := (I + cnT )−1(xn+ en+1).

(6) (iii) Select a relaxation param-eter αn ∈ [0, 1] and compute the

(n + 1)th iterate:

Algorithm 3.2

(i) Select x0 ∈ H arbitrarily.

(ii) Choose a regularization pa-rameter cn > 0 with error en+1 ∈

H and compute ˜

xn+1 := (I + cnT )−1(xn+ en+1).

(iii) Select a relaxation param-eter αn ∈ [0, 1] and compute the

(n + 1)th iterate:

xn+1 := αnx0+ (1 − αn)˜xn+1. (8)

Then we have the following con-vergence results for Algorithms 3.1 and 3.2. In order to achieve this, we need to impose the following condition:

ken+1k ≤ ηnk˜xn+1− xnk (9)

with P∞

n=0ηn2 < ∞.

Theorem 3.1. Let {xn}∞n=0 be

the sequence generated by Algo-rithm 3.1.Assume that condition (9) is satisfied and that

(i) {αn} is bounded away form

1, namely 0 ≤ αn ≤ 1 − δ for some

δ ∈ (0, 1) ;

(ii) {cn}∞n=0 ⊂ [c, ∞) for some

c > 0.

Then the following statements are valid:

(a) there exists an integer N0 ≥

0 such that for all n ≥ N0

kxn+1− x∗k2 ≤ βkxn− x∗k2

−δ

2k˜xn+1− xnk 2

for all x∗ ∈ S where β = 1 + 2η2n

1−2η2 n. (b) limn→∞k˜xn+1− xnk = 0; (c) {xn} converges weakly to a point in S. Theorem 3.2. Let {xn}∞n=0 be

the sequence generated by Algo-rithm 3.2. Assume that condition (9) is satisfied and that

(i) limn→∞αn = 0,P∞n=0αn =

∞;

(ii)limn→∞cn = ∞.

Then {xn} converges strongly to

Ps(x0).

4. Project Evaluation

In this project, we introduced two new algorithms for finding approximate solutions of zeros of a maximal monotone operators. Some new existence convergence results for these algorithms were obtained. We believe the results obtained in this project makes sig-nificant contribution to the lit-erature. Part of the results of this project has been written as a full paper and has been submitted to some well known international journal for possible publication.

References

[1] S. Aizicovici and Y. Q. Chen, Note on the topological de-gree of the subdifferential of a lower semi-continuous convex function, Proc. Amer. Math. Soc., 126(1998), 2905-2908.

[2] H. Brezis, ”Operateurs Max-imaux Mono-tone et Semi-Groups de Con-tractions dans les Espaces de Hilbert”, North-Holland, Am-sterdam, 1973.

[3] R. S. Burachik, A. N. Iusem,and B. F. Svaiter, En-largement of monotone op-erators with applications to variational inequalities, Set-Valued Anal., 5(1997), 159-180.

[4] Y. Censor and S. A. Zenios, The proximal minimization algorithm with D-functions, J. Optim. Theory Appl., 73(1992), 451-464.

[5] G. Chen and M. Teboulle ,A proximal-based decompo-sition method for convex min-imization problems, Math. Programming, 64(1994), 81-101.

[6] Y. Q. Chen, A generalization of the zero point theorem of maximal monotone mappings, Acta Math. Sinica, 38(1995), 831-836.

[7] S. S.Chang and Y. J. Cho, et al., Generalized set-valued variational inclusions in Ba-nach spaces, J. Math. Anal. Appl., 246(2000), 409-422. [8] G. Cohen, Auxiliary problem

principle extended to varia-tional inequalities, J. Optim. Theory Appl., 59(1998), 325-333.

[9] P. J. da Silva e Silva, J. Eck-stein and C. Humes,” Rescal-ing and Stepsize Selection in Proximal Methods Using Sep-arable Generalized Distance, ”RUTCOR Research Report RRR 35-99, Rutgers Univer-sity, 1999.

[10] R. S. Dembo, S. C. Eisenstat and and T. Steihaug, Inex-act Newton Methods, SIMA J. Numer. Anal., 19(1982), 400-408.

[11] J. Eckstein, Nonlinear prox-imal point algorithms using Bregman functions, with ap-plications to convex program-ming, Math. Oper. Res., 18(1993). 202-226.

[12] J. Eckstein and D. P. Bertsekas, On the Douglas-Rachford splitting method and the proximal points algo-rithm for maximal monotone operators, Math. Program-ming, 55(1992), 293-318. [13] J. Eckstein,

Approx-imate iterations in Bregman-function-based proximal algo-rithms, Math. Programming, 83(1998), 113-123.

[14] D. R. Han and B. S. He, A new accuracy criterion for ap-proximate proximal point al-gorithms, J. Math. Anal. Appl., 263(2001), 343-354. [15] B. S. He, Inexact implicit

methods for monotone general variational inequalities,Math. Programming, 86(1999), 199-217.

[16] I. Ekeland and R. Temam, ”Convex Analysis and Vari-ational Problems”, North-Holland, Amsterdam, 1973. [17] K. C. Kiwiel, Proximal

min-imization methods with gen-eralized Bregman functions, SIAM J. Control Optim., 35(1997), 1142-1168.

[18]

M. Kyono and M. Fukushima,

Nonlinear proximal decompo-sition method for convex pro-gramming, J. Optim. theory Appl., 106(2000), 357-372. [19] J. S. Pang, Inexact

New-ton methods for the non-linear complementarity prob-lem, Math. Programming, 36(1986), 54-71.

[20] R. T. Rockafellar, Mono-tone operators and the prox-imal point algorithm, SIAM J. Control Optim., 14(1976), 877-898.

[21] M. V. Solodov and B. F. Svaiter, A hybrid projection-proximal point algorithm, J. Convex Anal., 6(1999), 59-70. [22] M. V. Solodov and B. F. Svaiter, A hybrid approxi-mate extragradient-proximal point algorithm us-ing the enlargement of a max-imal monotone operator, Set-valued Anal., 7(1999), 323-345.

[23] M. V. Solodov and B. F. Svaiter, An inexact hybrid generalized proximal point al-gorithm and some new re-sult on the theory of Bregman functions, Math. Oper. Res., 25(2000), 214-230.

[24] K. K. Tan and H. K. Xu, Approximating fixed points of nonexpansive mappings by the Ishikawa iteration pro-cess, J. Math. Anal. Appl., 178(1993), 301-308.

[25] M. Teboulle, Convergence of proximal-like algorithms, SIAM J. Optim., 7(1997), 1069-1083 .

[26] H. K. Xu, Iterative algo-rithms for nonlinear opera-tors, J. London Math. Soc., 66(2)(2002), 240-256.

[27] E. H. Zarantonello, Projec-tions on convex sets in Hilbert space and spectral theory, in: Contributions to Nonlinear Functional Analysis(edited by E.H.Zarantonello), Academic Press, New York, NY, 1971.