MANAGEMENT OPTIMIZATION
Volume 9, Number 1, January 2013 pp. 153–169
PROXIMAL POINT ALGORITHM FOR NONLINEAR COMPLEMENTARITY PROBLEM BASED ON THE GENERALIZED FISCHER-BURMEISTER MERIT FUNCTION
Yu-Lin Chang and Jein-Shan Chen
Department of Mathematics National Taiwan Normal University
Taipei 11677, Taiwan
Jia Wu
School of Mathematical Sciences Dalian University of Technology
Dalian 116024, China
(Communicated by Liqun Qi)
Abstract. This paper is devoted to the study of the proximal point algorithm for solving monotone and nonmonotone nonlinear complementarity problems.
The proximal point algorithm is to generate a sequence by solving subproblems that are regularizations of the original problem. After given an appropriate cri- terion for approximate solutions of subproblems by adopting a merit function, the proximal point algorithm is verified to have global and superlinear con- vergence properties. For the purpose of solving the subproblems efficiently, we introduce a generalized Newton method and show that only one New- ton step is eventually needed to obtain a desired approximate solution that approximately satisfies the appropriate criterion under mild conditions. The motivations of this paper are twofold. One is analyzing the proximal point al- gorithm based on the generalized Fischer-Burmeister function which includes the Fischer-Burmeister function as special case, another one is trying to see if there are relativistic change on numerical performance when we adjust the parameter in the generalized Fischer-Burmeister.
1. Introduction. In the last decades, people have put a lot of their energy and attention on the complementarity problem due to its various applications in oper- ation research, economics, and engineering, see [16, 18,30] and references therein.
The nonlinear complementarity problem (NCP) is to find a point x ∈ IRnsuch that NCP(F ) : hF (x), xi = 0, F (x) ∈ IRn+, x ∈ IRn+, (1) where F : IRn→ IRnis a continuously differentiable mapping with F := (F1, F2, . . . , Fn)T. Many solution methods have been developed to solve NCP(F) [3, 4, 18, 19, 20, 21, 22, 24, 30, 40, 41]. For more details, please refers to the excellent monograph [14]. One of the most popular methods is to reformulate the NCP(F) as
2010 Mathematics Subject Classification. 49K30, 65K05, 90C33.
Key words and phrases. Complementarity problem, proximal point algorithm, approximation criterion.
Corresponding author: J.-S. Chen, Member of Mathematics Division, National Center for Theoretical Sciences, Taipei Office. Research of J. Chen is supported by National Science Council of Taiwan.
153
a unconstrained optimization problem and then to solve the reformulated problem by the unconstrained optimization technique. This kind of methods is called the merit function approach, where the merit functions are usually constructed via some NCP-functions.
A function φ : IR2→ IR is called an NCP-function if it satisfies φ(a, b) = 0 ⇐⇒ a ≥ 0, b ≥ 0, ab = 0.
Furthermore, if φ(a, b) ≥ 0 for all (a, b) ∈ IR2 then the NCP-function φ is called a nonnegative NCP-function. In addition, if a function Ψ : IRn → IR+ satisfying Ψ(x) = 0 if and only if x solves the NCP, then Ψ is called a merit function for the NCP. How to construct a merit function via an NCP-function? If φ is an NCP- function, an easy way to construct a merit function is defining Ψ : IRn → IR+
by
Ψ(x) :=
n
X
i=1
1
2φ2(xi, Fi(x)).
With this merit function, finding a solution of the NCP is equivalent to seeking a global minimum of the unconstrained minimization problem
x∈IRminnΨ(x)
with optimal value zero. In fact, many NCP-functions have been proposed in the literature. Among them, the Fischer-Burmeister (FB) function is one of the most popular NCP-functions, which is defined as
φFB(a, b) := a + b −p
a2+ b2, ∀(a, b) ∈ IR2.
One generalization of FB function was given by Kanzow and Kleinmichel in [23]:
φθ(a, b) := a + b −p
(a − b)2+ θab, θ ∈ (0, 4), ∀(a, b) ∈ IR2. (2) Another generalization proposed by Chen [3, 4, 5] and called generalized Fischer- Burmeister function is defined as
φp(a, b) := a + b −p|a|p p+ |b|p, p ∈ (1, ∞), ∀(a, b) ∈ IR2. (3) Among all various methods for solving the NCP, we focus on the proximal point algorithm (PPA) in this paper. The PPA is known for its theoretically nice con- vergence properties, which was first proposed by Martinet [27] and further studied by Rockafellar [35], and was originally designed for finding a vector z satisfying 0 ∈ T (z) where T is a maximal monotone operator. Therefore, it can be applied to a broad class of problems such as convex programming problems, monotone vari- ational inequality problems, and monotone complementarity problems. In general, PPA generates a sequence by solving subproblems that are regularizations of the original problem. More specifically, for the case of monotone NCP(F ), given the current point xk, PPA obtains the next point xk+1 by approximately solving the subproblem
NCP(Fk) : hFk(x), xi = 0, Fk(x) ∈ IRn+, x ∈ IRn+, (4) where Fk: IRn→ IRn is defined by
Fk(x) := F (x) + ck(x − xk) with ck> 0. (5) It is obvious that Fk is strongly monotone when F is monotone. Then, by [14, Theorem 2.3.3], the subproblem NCP(Fk), which is more tractable than the orig- inal problem, always has a unique solution. Thus, PPA is well-defined. It was
pointed out in [26, 35] that with appropriate criteria for approximate solutions of subproblems (4), PPA has global and superlinear convergence property under mild conditions. Another implementation issue is how to solve subproblems efficiently and obtain an approximate solution such that the approximation criterion for the subproblem is fulfilled. A generalized Newton method proposed by De Luca et al.
[25] which is used to solve subproblems. The approximation criterion under given conditions is eventually approximately fulfilled by a single Newton iteration of the generalized Newton method. As for the case of nonmonotone NCP, similar idea was also employed for solving P0-NCP in [43].
In this paper, we look into again the PPA for monotone NCP and P0-NCP studied in [42] and [43], respectively. We consider the PPA based on different NCP- functions, indeed based on the class of φp as in (3). Analogous theoretical analysis for PPA based on φp can be established easily. However, this is not the main purpose of doing such extension. In fact, there have been reported [3,4,6,7] that changing the value of p has various influence on numerical performance for different types of algorithms. It is our intension to know whether such phenomenon occurs when employing PPA method for solving NCPs. As will be seen in Section 5, such phenomenon does not appear in PPA, more specifically, changing the parameter p does change the numerical performance for the proposed PPA, however, such change does not depend on p regularly. This together with earlier reports offer us a further understanding about the generalized Fischer-Burmeister function.
2. Preliminaries. In this section, we review some background materials that will be used in the sequel and briefly introduce the proximal point algorithm.
2.1. Mathematical concepts. Given a set Ω ∈ IRn locally closed around ¯x ∈ Ω, the regular normal cone to Ω at ¯x is defined as
NbΩ(¯x) :=n
v ∈ IRn
lim sup
x→¯Ωx
hv, x − ¯xi
||x − ¯x|| ≤ 0o . The (limiting) normal cone to Ω at ¯x is set to be
NΩ(¯x) := lim sup
x→¯Ωx
NbΩ(x),
where “limsup” is the Painlev´e-Kuratowski outer limit of sets, see [36]. If Ω is the nonegative orthant IRn+, the normal cone to Ω at ¯x is
NIRn+(¯x) =
{y ∈ IRn|h¯x − z, yi ≥ 0, ∀z ≥ 0} if ¯x ≥ 0,
∅ otherwise.
We now recall definitions of various monotonicity and P -properties of a mapping which are needed for subsequent analysis.
Definition 2.1. Let F : IRn → IRn.
(a): F is said to be monotone if (x − y)T(F (x) − F (y)) ≥ 0 for all x, y ∈ IRn. (b): F is said to be strongly monotone with modulus µ > 0 if (x − y)T(F (x) −
F (y)) ≥ µkx − yk2for all x, y ∈ IRn.
(c): F is said to be an P0-function if for all x, y ∈ IRn with x 6= y there is an index i such that xi6= yi and (xi− yi)[Fi(x) − Fi(y)] ≥ 0.
(d): F is said to be an P -function if for all x, y ∈ IRn with x 6= y there is an index i such that (xi− yi)[Fi(x) − Fi(y)] > 0.
It is well known that, when F is continuously differentiable, F is monotone if and only if ∇F (ζ) is positive semidefinite for all ζ ∈ IRn while F is strongly monotone if and only if ∇F (ζ) is positive definite for all ζ ∈ IRn. For more details about monotonicity, please refer to [14]. In addition. it can be easily verified that if F is P0-function, then the function Fk defined by (5) is P -function and the Jacobian matrices of Fk are P -matrices. F is said to be uniformly Lipschitz continuous on a set Ω with modulus κ > 0 if kF (x) − F (y)k ≤ κkx − yk for all x, y ∈ Ω.
Moreover, for a vector-valued Lipschitz continuous mapping F : IRn → IRm, the B-subdifferential of F at x denoted by ∂BF (x) is defined as
∂BF (x) :=
lim
k→∞J F (xk)
xk → x, F is differentiable at xk
.
The convex hull of ∂BF (x) is the Clarke’s generalized Jacobian of F at x denoted by ∂F (x), see [10]. We say that F is strongly BD-regular at x if every element of
∂BF (x) is nonsingular.
We need another important concept named semismoothness which was first in- troduced in [28] for functionals and was further extended in [33] to vector-valued functions. Let F : IRn → IRm be a locally Lipschitz continuous mapping. We say that F is semismooth at a point x ∈ IRn if F is directionally differentiable and for any ∆x ∈ IRn and V ∈ ∂F (x + ∆x) with ∆x → 0, there has
F (x + ∆x) − F (x) − V (∆x) = o(||∆x||).
Furthermore, F is said to be strongly semismooth at x if F is semismooth at x and for any ∆x ∈ IRn and V ∈ ∂F (x + ∆x) with ∆x → 0, there holds
F (x + ∆x) − F (x) − V (∆x) = O(||∆x||2).
To close this subsection, we introduce the R-regularity of a solution ¯x to NCP.
For a solution ¯x ∈ IRn to the NCP(F ), we define the following three index sets which are associated with ¯x:
α := { i | ¯xi> 0, Fi(¯x) = 0}, β := { i | ¯xi= Fi(¯x) = 0}, γ := { i | ¯xi= 0, Fi(¯x) > 0}.
We say that the solution ¯x is R-regular [34] if ∇Fαα(¯x) is nonsingular and the Schur complement of ∇Fαα(¯x) in
∇Fαα(¯x) ∇Fαβ(¯x)
∇Fβα(¯x) ∇Fββ(¯x)
is an P -matrix.
2.2. Complementarity and merit functions. Back to the generalized FB func- tion φp, it has been proved in [3,4,5,6] that the function φp given in (3) possess a system of favorite properties, such as strong semismoothness, Lipschitz continuity, and continuous differentiability except for the point (0, 0). Due to these favorable properties, given a certain mapping ˆF : IRn → IRn, the NCP( ˆF ) can be reformu- lated as the following nonsmooth system of equations:
Φp(x) :=
φp(x1, ˆF1(x)) ... φp(xi, ˆFi(x))
... φp(xn, ˆFn(x))
= 0. (6)
It was also shown that the squared norm φp which is given by
ψp(a, b) := |φp(a, b)|2 (7) is continuously differentiable. Moreover, the merit functions Ψp induced from ψp
Ψp(x) := kΦp(x)k2=
n
X
i=1
|φp(x, ˆFi(x))|2=
n
X
i=1
ψp(xi, ˆFi(x)) (8)
possess SC1 property (i.e., they are continuously differentiable and their gradients are semismooth) and LC1 property (i.e., they are continuously differentiable and their gradients are Lipschitz continuous) under suitable assumptions. Note that Φp(x) is not differentiable at x when xi = ˆFi(x) = 0 for some 1 ≤ i ≤ n. In addition, in light of [8, Lemma 3.1], we know that an element V of ∂BΦp(x) can be expressed as
V = Da+ Db∇ ˆF (x)T, (9)
where Da and Db are diagonal matrices given by
((Da)ii, (Db)ii) (10)
= (
1 − sgn(xi)|xi|p−1
k(xi, ˆFi(x))kp−1p
, 1 −sgn( ˆFi(x))| ˆFi(x)|p−1
k(xi, ˆFi(x))kp−1p
if (xi, ˆFi(x)) 6= (0, 0),
(1 − η, 1 − ξ) otherewise,
with k(xi, ˆFi(x))kp−1p =
p
q
|xi|p+ | ˆFi(x)|p
p−1
and (η, ξ) being a vector satisfying
|η|p−1p + |ξ|p−1p = 1.
Lemma 2.2. Let Da and Db be defined as in (10). Then, the following hold.
(a): The diagonal matrices Da and Db satisfy (Da)ii+ (Db)ii ≥ 2 −√p
2, ∀i = 1, 2, · · · , n.
(b): Suppose that ˆF is strongly monotone with modulus µ. Then, k(Da+ Db∇ ˆF (x)T)−1k ≤ 1
B1µ where B1= 2−p
√ 2
n max{1, k∇ ˆF (x)k}.
Proof. (a) To proceed, we discuss two cases as below.
(i) If (xi, ˆFi(x)) 6= (0, 0), then
(Da)ii = 1 − sgn(xi)|xi|p−1 k(xi, ˆFi(x))kp−1p
(Da)ii = 1 −sgn( ˆFi(x))| ˆFi(x)|p−1 k(xi, ˆFi(x))kp−1p
. From the arguments of [7, Lemma 3.3], there proves an inequality
|a|p−1+ |b|p−1 k(a, b)kp−1p
≤ 21p for p > 1.
With this inequality, it is clear that (Da)ii+ (Db)ii≥ 2 −√p
2 under this case.
(ii) If (xi, ˆFi(x)) = (0, 0), then ((Da)ii, (Db)ii) = (1 − η, 1 − ξ) where (η, ξ) satisfy
|η|p−1p + |ξ|p−1p = 1. To prove the desired result, it is equivalent to showing η + ξ ≤
√p
2. This can be seen just by applying the H¨older inequality η + ξ ≤ (1p+ 1p)1p(|η|q+ |ξ|q)1q
where 1p+1q = 1 (note that q =p−1p ). Thus, the proof is complete.
(b) The arguments are similar to [42, Corollary 2.7] which can be obtained by applying part(a) and [42, Proposition 2.6].
With the expression of ∂BΦp(x), it is now possible to provide sufficient conditions for the strongly BD-regularity of Φpat a solution of the nonlinear complementarity problem. Let ¯x be a solution to NCP( ˆF ). The following statements indicate under what conditions Φp is strongly BD-regular at ¯x. These results are important from the algorithmic point of view.
Lemma 2.3. Let ¯x be a solution to NCP( ˆF ). Suppose that either ¯x is an R-regular solution or ∇ ˆF (¯x) is an P -matrix. Then every matrix in ∂BΦp(¯x) is nonsingular, i.e., Φp is strongly BD-regular at ¯x.
Proof. The proofs are similar to those in [15, Propositon 3.2] and [22, Corollary 22, Corollary 23], we omit them here.
Next, we talk about another merit function Ψ : IRn → IR+ which is utilized in the PPA method studied in [42] and provides a more favorable error bound than Ψp:
Ψ(x) :=
n
X
i=1
n|xiFˆi(x)| + |φNR(xi, ˆFi(x))|o , where the natural residual function φNR : IR2→ IR is defined by
φNR(a, b) := min{a, b}.
It is clear that Ψ(x) ≥ 0, and Ψ(x) = 0 if and only if x is a solution of NCP( ˆF ).
The below lemma shows the error bound property for such function Ψ.
Lemma 2.4. [42, Lemma2.11] Suppose that ˆF is strongly monotone with modulus µ. Then we have
kx − ˆxk ≤ 2 max{1, kxk}
s Ψ(x)
µ for all x ∈n
y ∈ IRn+ | Ψ(y) ≤ µ 4 o
where ˆx is the unique solution of NCP( ˆF ).
Now, we here present some basic properties regarding generalized Fischer-Burmei- ster function from [7] which we will be used later.
Lemma 2.5. [7, Lemma 3.1, Proposition 3.1] The functions ψp(a, b) and Ψp(x) defined by (7) and (8) have the following favorable properties:
(a): ψp(a, b) is an NCP-function.
(b): ψp(a, b) is continuously differentiable everywhere.
(c): For p ≥ 2, the gradient of ψp(a, b) is Lipschitz continuous on any nonempty bounded set.
(d): ∇aψp(a, b) · ∇bψp(a, b) ≥ 0 for any (a, b) ∈ IR2, and the equality holds if and only if ψp(a, b) = 0.
(e): ∇aψp(a, b) = 0 ⇐⇒ ∇bψp(a, b) = 0 ⇐⇒ ψp(a, b) = 0.
(f ): (2−√p
2)2ΨNR(x) ≤ Ψp(x) ≤ (2+√p
2)2ΨNR(x), where ΨNR(x) =Pn i=1φ2
NR(xi, Fˆi(x)).
Consequently, analogous to [42, Lemma 2.8, Lemma 2.9] and [43, Proposition 2.2], the following results can be achieved immediately.
Lemma 2.6. The mappings Φp and Ψp defined in (6) and (8) have the following properties.
(a): If ˆF is continuously differentiable, then Φp is semismooth.
(b): If ∇ ˆF is locally Lipschitz continuous, then Φp is strongly semismooth.
(c): If ˆF is continuously differentiable, then Ψp is continuously differentiable everywhere.
(d): If ˆF is monotone, then any stationary point of Ψpis a solution of NCP( ˆF ).
(e): If ˆF is an P0-function, then every stationary point of Ψp is a solution of NCP( ˆF ).
(f ): If ˆF is strongly monotone with modulus µ and Lischitz continuous with constant L, thenpΨp(x) provides a global error bound for NCP( ˆF ), that is,
kx − ˆxk ≤B2(L + 1) µ
q
Ψp(x), for all x ∈ IRn,
where ˆx is the unique solution of NCP( ˆF ) and B2 is a positive constant inde- pendent of ˆF .
(g): Let S ⊂ IRnbe a compact set. Suppose ˆF is strongly monotone with modulus µ and Lischitz continuous with constant L on S. Then pΨp(x) provides a global error bound on S, that is, there exists a positive constant B2 such that
kx − ˆxk ≤ B2(L + 1) µ
q
Ψp(x), for all x ∈ S, where ˆx is the unique solution of NCP( ˆF ).
Proof. Part(a) and (b) are clear since φp(a, b) is strongly semismooth. Part(c) is implied by Lemma2.5(b) while Lemma2.5(d)-(e) yield part(d). Part(e) is from [5, Proposition 3.4]. From Lemma2.5(f), we have
pΨNR(x) ≤ 1 2 −√p
2 q
Ψp(x).
This together with [29, Theorem 3.1] gives kx − ˆxk ≤(L + 1)
µ
pΨNR(x) ≤ (L + 1) (2 −√p
2)µ q
Ψp(x), where ΨNR(x) =Pn
i=1φ2
NR(xi, ˆFi(x)). This completes part(f) and part(g).
2.3. Proximal point algorithm. Let T : IRn ⇒ IRn be a set-valued mapping defined by
T (x) := F (x) + NK(x). (11)
It is known that T is a maximal monotone mapping and NCP(F ) is equivalent to the problem of finding a point x such that 0 ∈ T (x). Then, for any starting point x0, PPA generates a sequence {xk} by the approximate rule:
xk+1≈ Pk(xk),
where Pk := I +c1
kT−1
is a vector-valued mapping from IRn to IRn, {ck} is some sequence of positive real numbers, and xk+1≈ Pk(xk) means that xk+1is an approximation to Pk(xk). Accordingly, Pk(xk) is given by
Pk(xk) =
I + 1
ck
(F + NK)
−1
(xk), from which we have
Pk(xk) ∈ SOL(NCP(Fk)),
where Fkis defined by (5) and SOL(NCP(Fk)) denotes the solution set of NCP(Fk).
Therefore, xk+1 is obtained by an approximate solution of NCP(Fk). As remarked in [42], when ck is small, the subproblem is close to the original one whereas when ck is large, a solution of the subproblem is expected to lie near xk. Moreover, to ensure convergence of PPA, xk+1 must be located sufficiently near the solution Pk(xk). Among others, two general criteria for the approximate calculation of Pk(xk) proposed by Rockafellar [35] are as follows:
Criterion 1.:
xk+1− Pk(xk) ≤ εk,
∞
X
k=0
εk< ∞.
Criterion 2.:
xk+1− Pk(xk)
≤ ηkkxk+1− xkk,
∞
X
k=0
ηk< ∞.
As mentioned in [26, 35], there says the above Criterion 1 guarantees global convergence while Criterion 2, which is rather stringent, ensures superlinear con- vergence. The following two theorems elaborate this issue.
Theorem 2.7. [35, Theorem 1] Let {xk} be any sequence generated by the PPA under Criterion 1 with {ck} bounded. Suppose NCP(F ) has at least one solution.
Then {xk} converges to a solution x∗ of NCP(F ).
Theorem 2.8. [26, Theorem 2.1] Suppose the solution set ¯X of NCP(F ) is nonemp- ty, and let {xk} be any sequence generated by PPA with Criteria 1 and Criterion 2 and ck → 0. Let us also assume that
∃ δ > 0, ∃ C > 0, s.t. dist(x, ¯X) ≤ Ckωk whenever x ∈ T−1(ω) and kωk ≤ δ. (12) Then the sequence {dist(xk, ¯X)} converges to 0 superlinearly.
3. A proximal point algorithm for the monotone NCP.
3.1. A proximal point algorithm. Based on the previous discussion, in this subsection we describe PPA for solving NCP(F ) where F is smooth and monotone.
We first illustrate the related mappings that will be used in the remainder of this paper.
Here the mappings Φp and Ψp are the same as the ones defined by (6) and (8), respectively, except the mapping ˆF is substituted by F , i.e.,
Φkp(x) :=
φp(x1, F1k(x)) ... φp(xi, Fik(x))
... φp(xn, Fnk(x))
, Φp(x) :=
φp(x1, F1(x)) ... φp(xi, Fi(x))
... φp(xn, Fn(x))
,
Ψkp(x) := kΦkp(x)k2, Ψp(x) := kΦp(x)k2, and Ψk(x) :=Pn
i=1ψ(xi, Fik(x)) with ψ(xi, Fik(x)) = |xiFik(x)| +
φNR(xi, Fik(x)) . Now we are in a position to describe the proximal point algorithm for solving NCP(F ).
Algorithm 3.1.
Step 0.: Choose parameters α ∈ (0, 1), c0∈ (0, 1) and an initial point x0∈ IRn. Set k := 0.
Step 1.: If xk satisfies Ψp(xk) = 0, then stop.
Step 2.: Let Fk(x) = F (x) + ck(x − xk). Get an approximation solution ˜xk+1 of NCP(Fk) that satisfies the conditions
Ψk [˜xk+1]+ ≤ c3kmin{1, kxk− ˜xk+1k}
4 max{1, k[˜xk+1]+k}2 . (13) Step 3.: Set xk+1:= [˜xk+1]+, ck+1= αck and k := k + 1. Go to Step 1.
The condition (13) is different from what was used in [42]. In fact, it can be decomposed into two parts as below
Ψk [˜xk+1]+
≤ c3k
4 max{1, k[˜xk+1]+k}2 (14) Ψk [˜xk+1]+
≤ c3kkxk− ˜xk+1k
4 max{1, k[˜xk+1]+k}2 (15) which corresponds to the aforementioned Criterion 1 and Criterion 2, respectively, to ensure the global and superlinear convergence. However, as shown in the next lemma, the condition (15) implies
q
Ψkp(˜xk+1) ≤ c3kkxk− ˜xk+1k. (16) Note that condition (14) and condition (16) are exactly what were used in [42]. In view of this, our modified PPA is more neat and compact.
Lemma 3.1. Suppose that the following inequality holds Ψk([˜xk+1]+) ≤ c3kkxk− ˜xk+1k
4 max{1, k[˜xk+1]+k}2. Then, we have
q
Ψkp(˜xk+1) ≤ c3kkxk− ˜xk+1k.
Proof. First, applying Lemma2.5(f) gives q
Ψkp(˜xk+1) ≤ (2 + √p
2)pΨNR(˜xk+1)
≤ (2 +√p 2)
n
X
i=1
|φNR(˜xk+1i , Fik(˜xk+1))|
≤ 4Ψk([˜xk+1]+).
On the other hand, from the assumption
Ψk([˜xk+1]+) ≤ c3kkxk− ˜xk+1k 4 max{1, k[˜xk+1]+k}2
and max{1, k[˜xk+1]+k}2≥ 1, we have
4Ψk([˜xk+1]+) ≤ c3kkxk− ˜xk+1k.
Thus, the desired result follows.
Theorem 3.2. Let ¯X be the solution set of NCP(F). If ¯X 6= ∅, then the sequence {xk} generated by Algorithm 3.1 converges to a solution x∗ of NCP(F).
Proof. Since Fk is strongly monotone with modulus ck > 0, Pk(xk) is the unique solution of NCP(Fk). From (13), we know
Ψk(xk+1) ≤ c3k
4 max{1, kxk+1k}2 ≤ck 4 . Then, it follows from Lemma2.4that
kxk+1− Pk(xk)k ≤ 2 max{1, kxk+1k}
rΨk(xx+1)
ck , (17)
which together with (13) implies
kxk+1− Pk(xk)k ≤ ck. (18) Thus, {xk} satisfies Criterion 1 and the global convergence is guaranteed in light of Theorem2.7.
In order to obtain superlinear convergence, we need the following assumption which is connected to the condition adopted in Theorem2.8(see Theorem3.4).
Assumption 3.1. k min{x, F (x)}k provides a local error bound for NCP(F ), i.e., there exist positive constants ¯δ and ¯C such that
dist(x, ¯X) ≤ ¯Ck min{x, F (x)}k for all x with k min{x, F (x)}k ≤ ¯δ, (19) where ¯X denotes the solution set of NCP(F ).
Lemma 3.3. [31, Proposition 3] If a Lipschitz continuous mapping H is strongly BD-regular at x∗, then there is a neighborhood N of x∗ and a positive constant α such that ∀x ∈ N and V ∈ ∂BH(x), V is nonsingular and kV−1k ≤ α. Furthermore, if H is semismooth at x∗ and H(x∗) = 0, then there exists a neighborhood N0 of x∗ and a positive constant β such that ∀x ∈ N0, kx − x∗k ≤ βkH(x)k.
We note that when ∇F (¯x) is positive definite at one solution ¯x of NCP(F ), we see that Assumption 3.1 is satisfied due to Lemma2.3and Lemma3.3in which we view H(x) = Φp(x). Lemma 3.3also indicates under what conditions Assumption 3.1 holds. Now, we show that Assumption 3.1 indeed implies the condition (12).
Theorem 3.4. Let T be defined by (11). If ¯X 6= ∅, then Assumption 3.1 implies condition (12), that is, there exist δ > 0 and C > 0 such that
dist(x, ¯X) ≤ Ckωk, whenever x ∈ T−1(ω) and kωk ≤ δ.
Proof. For all x ∈ T−1(ω) we have
ω ∈ T (x) = F (x) + NK(x).
Therefore there exists v ∈ NK(x) such that ω = F (x) + v. Because K is a convex set, it is easy to obtain
ΠK(x + v) = x. (20)
Noting that the projective mapping onto a convex set is nonexpansive, it follows from (20) that
kx − ΠK(x − F (x))k = kΠK(x + v) − ΠK(x − F (x))k ≤ kv + F (x)k = kωk.
From Assumption 3.1 and letting C = ¯C, δ = ¯δ yield the desired condition (12) in Theorem2.8.
The following theorem gives the superlinear convergence of Algorithm 3.1, whose proof is based on Theorem 3.4 and can be obtained in the same way as done in Theorem3.2.
Theorem 3.5. Suppose that Assumption 3.1 holds. Let {xk} be generated by Al- gorithm 3.1. Then the sequence {dist(xk, ¯X)} converges to 0 superlinearly.
Proof. By Theorem3.2, {xk} is bounded. Hence Fkis uniformly Lipschitz continu- ous with modulus L on a bounded set containing {xk}. By Lemma2.4and Lemma 3.1we have
k˜xk+1− Pk(xk)k ≤ B2(L + 1) ck
q
Ψkp(˜xk+1) ≤ B2(L + 1)c2kkxk− ˜xk+1k, for some positive constant B2. Then the desired results follows by applying Theorem 2.8and Theorem3.4.
3.2. Generalized Newton method. Although we have obtained the global and superlinear convergence properties of Algorithm 3.1 under mild conditions, we still need to care about how to obtain an approximation solution of the strongly mono- tone complementarity problem in its step 2 and what is the cost to ensure that Algorithm 3.1 is practically efficient. We will discuss this issue in this subsection.
More specifically, we introduce the generalized Newton method proposed by De Luca, Facchinei, and Kanzow [25] for solving the subproblems in Step 2 of Algo- rithm 3.1. As mentioned earlier, for each fixed k, Problem (4) is equivalent to the following nonsmooth equation
Φkp(x) = 0. (21)
We describe as below the generalized Newton method for solving the nonsmooth system (21), which is employed from what was introduced in [42] for solving NCP.
Algorithm 3.2 (generalized Newton method for NCP(Fk)).
Step 0.: Choose β ∈ (0,12) and an initial point x0∈ IRn. Set j := 0.
Step 1.: If kΦkp(xj)k = 0, then stop.
Step 2.: Select an element Vj ∈ ∂BΦkp(xj). Find the solution dj of the system
Vjd = −Φkp(xj). (22)
Step 3.: Find the smallest nonnegative integer ij such that Ψkp(xj+ 2−ijdj) ≤ (1 − β21−ij)Ψkp(xj).
Step 4.: Set xj+1:= xj+ 2−ijdj and j := j + 1. Go to Step 1.
Mimicking [25, Theorem 3.1], the following theorem which guarantees the con- vergence of the above algorithm can be obtained easily.
Theorem 3.6. Suppose that F is differentiable and strongly monotone and that ∇F is Lipschitz continuous around the unique solution ˆx of NCP(Fk). The Algorithm 3.2 globally converges to ˆx and the rate of convergence is quadratic.
4. A proximal point algorithm for the P0-NCP. Most ideas in the PPA for monotone NCP introduced in previous section can be adopted for the nonmonotone NCP. The main concern is that the global and superlinear convergence relying on the monotone properties cannot be carried over to nonomonotone case. Fortunately, for the P0-NCP which is a special subclass of nonmonotone NCPs, it is possible to execute PPA in such case with different conditions. This is what we will elaborate in this section. Note that it is known that if F is an P0-function, then Fk defined by (5) is an P -function. Therefore, the subproblem NCP(Fk) always has a unique solution, so that PPA is well-defined. Here, the definitions of mappings Φp, Ψp, Φkp and Ψkp are the same as those defined in Section 3. The PPA for solving NCP(F ) with P0-function can be described as follows:
Algorithm 4.1.
Step 0.: Choose parameters c0 > 0, δ0 ∈ (0, 1) and an initial point x0 ∈ IRn. Set k := 0.
Step 1.: If xk satisfies Ψp(xk) = 0, then stop.
Step 2.: Let Fk(x) = F (x) + ck(x − xk). Get an approximation solution ˜xk+1 of NCP(Fk) that satisfies the conditions
Ψkp(˜xk+1) ≤ 2δk2min{1, kxk− ˜xk+1k2}. (23) Step 3.: Set xk+1 := ˜xk+1. Choose ck+1 ∈ (0, ck) and δk+1 ∈ (0, δk). Set
k := k + 1. Go to Step 1.
We point it out that the condition (23) is different from the condition (13) used in Algorithm 3.1 because we are dealing with the nonmonotone NCP here. We first summarize some useful properties and give some assumptions that will be used in our convergence analysis.
Lemma 4.1. For any a, b and c, we have |φp(a, b + c) − φp(a, b)| ≤ 2|c|. Moreover, Φkp(x) − Φp(x)
≤ 2ckkx − xkk.
Proof. The proof is direct and an extension of [43, Lemma 2.2], we omit it here.
Proposition 1. Suppose that F is an P0-function. Then, for each k, the merit function Ψkp is coercive, i.e.,
lim
kxk→∞Ψkp(x) = +∞.
Proof. Based on the property of φp given by [5, Lemma 3.1], the conclusion can be drawn in a way similar to [13, Proposition 3.4].
The following two assumptions are needed for the nonmonotone NCP case.
Assumption 4.1. The sequence {ck} satisfies the following conditions:
(a): ck(xk+1− xk) → 0 if {xk} is bounded.
(b): ckxk→ 0 if {xk} is unbounded.
Assumption 4.2. The sequence {xk} converges to a solution x∗ of NCP(F ) that is strongly BD-regular with Φp.
A few words about these two assumptions. As remarked in [43], we can use the following update rules for parameters ck and δk in Algorithm 4.1 that satisfy both
Assumption 4.1 and step 3 in Algorithm 4.1.
ck = min
1, 1
kxkk2
min
(γ)k,1
2Ψp(xk)
, δk = (γ)k, (24) where γ ∈ (0, 1) is a given constant. Since F is an P0-function, from [12, Theorem 3.2] and [32, Proposition 2.5], we know that Assumption 4.2 ensures that x∗ is the unique solution of NCP(F ). In addition, sufficient conditions for Assumption 4.2 have already been given in Lemma2.3.
Now we are in a position to present the global and superlinear convergence for Algorithm 4.1 whose proofs actually can be shown in a way similar to [43, Theorem 1 and Theorem 3]. Thus, we omit them here.
Theorem 4.2. Suppose that F is an P0-function and the solution set of NCP(F) is nonempty and bounded. Suppose also that Assumption 4.1 holds. If δk → 0, then the sequence {xk} generated by Algorithm 4.1 is bounded and any accumulation point of {xk} is a solution of NCP(F ).
Theorem 4.3. Suppose that Assumption 4.1 and Assumption 4.2 hold and the solution set of NCP(F ) is bounded. Suppose also that δk → 0 and ck → 0. Then the sequence {xk} generated by Algorithm 4.1 converges superlinearly to the solution x∗ of NCP(F ).
Like the monotone case, for solving subproblem NCP(Fk), we also adopt the generalized Newton method, i.e., Algorithm 3.2 described in Section 3. Since F is an P0-function, it can be easily verified from the definition of Fk that the Jacobian matrices of Fk at any point x are P -matrices. Hence Φkp is strongly BD-regular at x by Lemma2.3. As a result, from [25, Theorem 11], we also have the following convergence theorem for Algorithm 3.2.
Theorem 4.4. Suppose that F is continuously differentiable and is an P0-function.
Then Algorithm 3.2 globally converges to the unique solution of NCP(Fk). Further- more, if ∇F is locally Lipschitz continuous, then the convergence rate is quadratic.
5. Numerical experiments. In this section, we report numerical results of Al- gorithm 3.1 and Algorithm 4.1 for solving NCP(F ) defined by (1). Our numerical experiments are carried out in MATLAB (version 7.8) running on a PC Intel core 2 Q8200 of 2.33GHz CPU and 2.00GB Memory for the test problems with all available starting points in MCPLIB [2].
In our numerical experiments, the stopping criteria for Algorithm 3.1 and Al- gorithm 4.1 are Ψp(xk) ≤ 1.0e − 8 and |(xk)T(F (xk))| ≤ 1.0e − 3. We also stop programs when the total iteration is more than 50. In Algorithm 3.1, we set the parameters as α = 0.5 and c0 = 0.5. In Algorithm 4.1, we adopt rules (24) to update the parameters where γ = 0.5. In Algorithm 3.2, we set the parameter β = 10−4 and the initial point for Newton’s method is selected as the current it- eration point in the main algorithm. Notice that the main task of Algorithm 3.2 for solving the subproblem, at each iterate, is to solve the linear system (22). In numerical implementation, we apply the preconditioner conjugate gradient square method for solving system (22). Because there is no description about how to dis- tinguish monotone NCP and non-monotone NCP from MCPLIB collection, we only use the iteration point to test the non-monotonicity. In summary, among 82 test problems in MCPLIB (different starting points are regarded as different test prob- lems), 50 test problems are verified to be non-monotone. As a result, Algorithm 4.1
is implemented on the whole 82 problems, while Algorithm 3.1 is only implemented on 32 problems which may be monotone. A good news is that with a proper p, all the test problems can be solved successfully, which shows that the proximal point algorithms for solving complementarity problems are efficient.
To present an objective evaluation and comparison of the performance of Algo- rithm 3.1 and Algorithm 4.1 with different p, we consider the performance profile introduced in [11] as a means. In particular, we regard Algorithm 3.1 or Algorithm 4.1 corresponding to a p as a solver, and assume that there are nssolvers and njtest problems from the MCPLIB collection J . We are interested in using the number of function evaluations as performance measure for Algorithm 3.1 or Algorithm 4.1 with different p. For each problem j and solver s, let
fj,s:= function evaluations required to solve problem j by solver s.
We compare the performance on problem j by solver s with the best performance by any one of the ns solvers on this problem; that is, we adopt the performance ratio
rj,s= fj,s
min{fj,s: s ∈S },
whereS is the set of four solvers. An overall assessment of each solver is obtained from
ρs(τ ) = 1 nj
size{j ∈J : rj,s≤ τ },
which is called the performance profile of the number of function evaluations for solver s. Note that ρs(τ ) approximates the probability for solver s that a perfor- mance ratio rj,s is within a factor τ of the best possible ratio.
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
The values of tau
The values of performance profile
p=1.1 p=2 p=5 p=50
Figure 1. Performance profile of function evaluations of Algo- rithm 3.1 with four p.
Figure 1 shows the performance profile of function evaluations in Algorithm 3.1 in the range of [0, 5] for four solvers on 32 test problems. The four solvers correspond to Algorithm 3.1 with p = 1.1, p = 2, p = 5 and p = 50, respectively. From this figure, we see that Algorithm 3.1 with p = 50 and p = 2 has the competitive wins (has the highest probability of being the optimal solver) and that the probabilities that it is the winner on a given monotone NCP are about 0.40 and 0.28, respectively. If we
choose being within a factor of greater than 2 of the best solver as the scope of our interest, then p = 2 and p = 1.1 would suffice, and the performance profile shows that the probability that Algorithm 3.1 with this p can solve a given monotone NCP in such range of the best solver is almost 100%. And Algorithm 3.1 with p = 5 has a comparable performance with p = 2 and p = 1.1 if we choose being within a factor of greater than 3 of the best solver as the scope of our interest. Although p = 50 has a competitive wins with p = 2, the probability that it can solve a given NCP within any positive factor of the best solver is lower than p = 2. Actually, p = 50 has the lowest probability within a factor of greater than 2 of the best solver. To sum up, Algorithm 3.1 with p = 2 have the best numerical performance than the others.
0 2 4 6 8 10 12 14 16 18 20
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
The values of tau
The values of performance profile
p=1.1 p=2 p=5 p=10 p=50
Figure 2. Performance profile of function evaluations of Algo- rithm 4.1 with five p.
Figure 2 shows the performance profile of function evaluations in Algorithm 4.1 in the range of [0, 20] for five solvers on 82 test problems. The five solvers correspond to Algorithm 4.1 with p = 1.1, p = 2, p = 5, p = 10 and p = 50, respectively.
From this figure, we see that Algorithm 4.1 with p = 1.1 has the best numerical performance (has the highest probability of being the optimal solver) and that the probability that it is the winner on a given NCP is about 0.37. If we choose being within a factor of 2 or 3 of the best solver, then p = 2 has a comparable performance with p = 1.1. If we choose being within a factor of greater than 11 or 16 of the best solver as the scope of our interest, the performance profile of p = 1.1 shows that the probabilities that Algorithm 4.1 with this p can solve a given NCP in such range of the best solver are 93% and 95%, respectively.
To sum up, although changing the parameter p does change the numerical perfor- mance for the proposed PPA, however, such change does not depend on p regularly, unlike mentioned in [3,4,6,7] which are reported for other algorithms.
Acknowledgments. The authors thank the referees for their careful reading of the paper and helpful suggestions.
REFERENCES
[1] E. D. Andersen, C. Roos and T. Terlaky, On implementing a primal-dual interior-point method for conic quadratic optimization, Mathematical Programming, 95 (2003), 249–277.
[2] S. C. Billups, S. P. Dirkse and M. C. Soares,A comparison of algorithms for large scale mixed complementarity problems, Computational Optimization and Applications, 7 (1997), 3–25.
[3] J.-S. Chen,The semismooth-related properties of a merit function and a descent method for the nonlinear complementarity problem, Journal of Global Optimization, 36 (2006), 565–580.
[4] J-.S. Chen, On some NCP-function based on the generalized Fischer-Burmeister function, Asia-Pacific Journal of Operational Research, 24 (2007), 401–420.
[5] J.-S. Chen and S.-H. Pan,A family of NCP functions and a desent method for the nonlinear complementarity problem, Computational Optimization and Applications, 40 (2008), 389–
404.
[6] J.-S. Chen and S.-H. Pan,A regularization semismooth Newton method based on the gen- eralized Fischer-Burmeister function for P0-NCPs, Journal of Computational and Applied Mathematics, 220 (2008), 464–479.
[7] J.-S. Chen, H.-T. Gao and S.-H. Pan,An R-linearly convergent derivative-free algorithm for the NCPs based on the generalized Fischer-Burmeister merit function, Journal of Computa- tional and Applied Mathematics, 232 (2009), 455–471.
[8] J.-S. Chen, S.-H. Pan and C.-Y. Yang, Numerical comparison of two effective methods for mixed complementarity problems, Journal of Computational and Applied Mathematics, 234 (2010), 667–683.
[9] X. Chen and H. Qi, Cartesian P -property and its applications to the semidefinite linear complementarity problem, Mathematical Programming, 106 (2006), 177–201.
[10] F. H. Clarke, “Optimization and Nonsmooth Analysis,” John Wiley and Sons, New York, 1983.
[11] E. D. Dolan and J. J. More, Benchmarking optimization software with performance profiles, Mathematical Programming, 91 (2002), 201–213.
[12] F. Facchinei,Structural and stability properties of P0 nonlinear complementarity problems, Mathematics of Operations Research, 23 (1998), 735–745.
[13] F. Facchinei and C. Kanzow,Beyond monotonicity in regularization methods for nonlinear complementarity problems, SIAM Journal on Control and Optimization, 37 (1999), 1150–
1161.
[14] F. Facchinei and J.-S. Pang, “Finite-Dimensional Variational Inequalities and Complemen- tarity Problems,” Springer Verlag, New York, 2003.
[15] F. Facchinei and J. Soares, A new merit function for nonlinear complementarity problems and a related algorithm, SIAM Journal on Optimazation, 7 (1997), 225–247.
[16] M. C. Ferris and J.-S. Pang, Engineering and economic applications of complementarity problems, SIAM J. Rev., 39 (1997), 669–713.
[17] A. Fischer,Solution of monotone complementarity problems with locally Lipschitzian func- tions, Mathematical Programming, 76 (1997), 513–532.
[18] P. T. Harker and J.-S. Pang,Finite dimensional variational inequality and nonlinear com- plementarity problem: a survey of theory, algorithms and applications, Mathematical Pro- gramming, 48 (1990), 161–220.
[19] Z.-H. Huang ,The global linear and local quadratic convergence of a non-interior continuation algorithm for the LCP, IMA J. Numer. Anal., 25 (2005), 670–684.
[20] Z.-H. Huang and W.-Z. Gu,A smoothing-type algorithm for solving linear complementarity problems with strong convergence properties, Appl. Math. Optim., 57 (2008), 17–29.
[21] Z.-H. Huang, L. Qi and D. Sun,Sub-quadratic convergence of a smoothing Newton algorithm for the P0- and monotone LCP, Mathematical Programming, 99 (2004), 423–441.
[22] H.-Y. Jiang, M. Fukushima and L. Qietal., A trust region method for solving generalized complementarity problems, SIAM Journal on Control and Optimization, 8 (1998), 140–157.
[23] C. Kanzow and H. Kleinmichel,A new class of semismooth Newton method for nonlinear complementarity problems, Computational Optimization and Applications, 11 (1998), 227–
251.
[24] C. Kanzow, N. Yamashita and M. Fukushima, New NCP-functions and their properties, Journal of Optimization Theory and Applications, 94 (1997), 115–135.
[25] T. De Luca, F. Facchinei and C. Kanzow, A semismooth equation approach to the solution of nonlinear complementarity problems, Mathematical Programming, 75 (1996), 407–439.
[26] F. J. Luque,Asymptotic convergence analysis of the proximal point algorithm, SIAM Journal on Control and Optimization, 22 (1984), 277–293.
[27] B. Martinet, Perturbation des m´ethodes d’opimisation, RAIRO Anal. Num´er., 12 (1978), 153–171.
[28] R. Mifflin, Semismooth and semiconvex functions in constrained optimization, SIAM Journal on Control and Optimization, 15 (1997), 957–972.
[29] J.-S. Pang, A posteriori error bounds for the linearly-constrained variational inequality prob- lem, Mathematics of Operations Research, 12 (1987), 474–484.
[30] J.-S. Pang, Complementarity problems, in “ Handbook of Global Optimization” (eds. R. Horst and P. Pardalos), Kluwer Academic Publishers, Boston, Massachusetts, (1994), 271–338.
[31] J.-S. Pang and L. Qi, Nonsmooth equations: Motivation and algorithms, SIAM Journal on Optimization, 3 (1993), 443–465.
[32] L. Qi,A convergence analysis of some algorithms for solving nonsmooth equations, Mathe- matics of Operations Research, 18 (1993), 227–244.
[33] L. Qi and J. Sun,A nonsmooth version of Newton’s method, Mathematical Programming, 58 (1993), 353–367.
[34] S. M. Robinson,Strongly regular generalized equations, Mathematics of Operations Research, 5 (1980), 43–62.
[35] R. T. Rockafellar,Monotone operators and the proximal point algorithm, SIAM Journal on Control and Optimization, 14 (1976), 877–898.
[36] R. T. Rockafellar and R. J-B. Wets, “Variational Analysis,” Springer-Verlag Berlin Heidelberg, 1998.
[37] D. Sun and L. Qi, On NCP-functions, Computational Optimization and Applications, 13 (1999), 201–220.
[38] D. Sun and J. Sun,Strong semismoothness of the Fischer-Burmeister SDC and SOC com- plementarity functions, Mathematical Programming, 103 (2005), 575–581.
[39] J. Wu and J.-S. Chen, A proximal point algorithm for the monotone second-order cone com- plementarity problem, Computational Optimization and Applications, 51 (2012), 1037–1063.
[40] N. Yamashita and M. FukushimaOn stationary points of the implicitm Lagrangian for nonlin- ear complementarity problems, Journal of Optimization Theory and Applications, 84 (1995), 653–663.
[41] N. Yamashita and M. Fukushima Modified Newton methods for solving a semismooth refor- mulation of monotone complementarity problems, Mathematical Programming, 76 (1997), 469–491.
[42] N. Yamashita and M. Fukushima, The proximal point algorithm with genuine superlinear convergence for the monotone complementarity problem, SIAM Journal on Optimization, 11 (2000), 364–379.
[43] N. Yamashita, J. Imai and M. Fukushima, The proximal point algorithm for the P0 com- plementarity problem, in “Complementarity: Applications, Algorithms and Extensions” (eds.
M. C. Ferris et al.), Kluwer Academic Publisher, (2001), 361–379.
Received May 2011; revised May 2012.
E-mail address: ylchang@math.ntnu.edu.tw E-mail address: jschen@math.ntnu.edu.tw E-mail address: wujia@dlut.edu.cn
