to appear in Computational Optimization and Applications, 2010
Neural networks for solving second-order cone constrained variational inequality problem
Juhe Sun 1 School of Science
Shenyang Aerospace University Shenyang 110136, China
E-mail: juhesun@abel.math.ntnu.edu.tw
Jein-Shan Chen 2 Department of Mathematics National Taiwan Normal University
Taipei 11677, Taiwan E-mail: jschen@math.ntnu.edu.tw
Chun-Hsu Ko
Department of Electrical Engineering I-Shou University
Kaohsiung 840, Taiwan E-mail: chko@isu.edu.tw
April 18, 2010 (revised on July 12, 2010)
Abstract. In this paper, we consider using the neural networks to efficiently solve the second-order cone constrained variational inequality (SOCCVI) problem. More specifi- cally, two kinds of neural networks are proposed to deal with the Karush-Kuhn-Tucker (KKT) conditions of the SOCCVI problem. The first neural network uses the Fischer- Burmeister (FB) function to achieve an unconstrained minimization which is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit function is a Lyapunov function and this neural network is asymptotically stable. The second neural network is introduced for solving a projection formulation whose solutions coincide with the KKT triples of SOCCVI problem. Its Lyapunov stability and global convergence
1also affiliated with Department of Mathematics, National Taiwan Normal University.
2Member of Mathematics Division, National Center for Theoretical Sciences, Taipei Office. The author’s work is partially supported by National Science Council of Taiwan.
are proved under some conditions. Simulations are provided to show effectiveness of the proposed neural networks.
Key words. Second-order cone, variational inequality, Fischer-Burmeister function, neu- ral network, Lyapunov stable, projection function.
1 Introduction
Variational inequality (VI) problem, which was introduced by Stampacchia and his col- laborators [19, 30, 31, 36, 37], has attracted much attention from researchers of engi- neering, mathematics, optimization, transportation science, and economics communities, see [1, 26, 27]. It is well known that VIs subsume many other mathematical problems, including the solution of systems of equations, complementarity problems, and a class of fixed point problems. For a complete discussion and history of the finite VI problem and its associated solution methods, we refer the interested readers to the excellent survey text by Facchinei and Pang [13], the monograph by Patriksson [35], the survey article by Harker and Pang [18], the Ph.D. thesis of Hammond [16] and the references therein.
In this paper, we are interested in solving the second-order cone constrained varia- tional inequality (SOCCVI) problem whose constraints involve the Cartesian product of second-order cones (SOCs). The problem is to find x ∈ C satisfying
hF (x), y − xi ≥ 0, ∀y ∈ C, (1)
where the set C is finitely representable as
C = {x ∈ Rn: h(x) = 0, −g(x) ∈ K}. (2) Here h·, ·i denotes the Euclidean inner product, F : Rn→ Rn, h : Rn→ Rl and g : Rn→ Rm are continuously differentiable functions and K is a Cartesian product of second-order cones (or Lorentz cones), expressed as
K = Km1 × Km2 × · · · × Kmp, (3) where l ≥ 0, m1, m2, · · · , mp ≥ 1, m1+ m2 + · · · + mp = m, and
Kmi := {(xi1, xi2, · · · , ximi)T ∈ Rmi | k(xi2, · · · , ximi)k ≤ xi1}
with k · k denoting the Euclidean norm and K1 the set of nonnegative reals R+. A special case of equation (3) is K = Rn+, namely the nonnegative orthant in Rn, which corresponds to p = n and m1 = · · · = mp = 1. When h is affine, an important special case of the SOCCVI problem corresponds to the KKT conditions of the convex second-order cone program (CSOCP):
min f (x)
s.t. Ax = b, −g(x) ∈ K, (4)
where A ∈ Rl×n has full row rank, b ∈ Rl, g : Rn → Rm and f : Rn → R. Furthermore, when f is a convex twice continuously differentiable function, problem (4) is equivalent to the following SOCCVI problem: Find x ∈ C satisfying
h∇f(x), y − xi ≥ 0, ∀y ∈ C, where
C = {x ∈ Rn: Ax − b = 0, −g(x) ∈ K}.
For solving the constrained variational inequalities and complementary problems (CP), many computational methods have been proposed, see [3, 4, 6, 8, 13, 40] and references therein. These methods include the method based on merit function, interior method, Newton method, nonlinear equation method, projection method and its variant versions. Another class of techniques for solving the VI problem exploits the fact that the KKT conditions of a VI problem comprise a mixed complementarity problem (MiCP), involving both equations and nonnegativity constraints. In other words, the SOCCVI problem can be solved by analyzing its KKT conditions:
L(x, µ, λ) = 0,
hg(x), λi = 0, −g(x) ∈ K, λ ∈ K, h(x) = 0,
(5)
where L(x, µ, λ) = F (x) + ∇h(x)µ + ∇g(x)λ is the variational inequality Lagrangian function, µ ∈ Rl and λ ∈ Rm. However, in many scientific and engineering applications, it is desirable to have a real-time solution for the VI and CP problems. Thus, at present, for solving the VI and CP problems, many researchers employ the neural network method which is a promising way to overcome this problem.
Neural networks for optimization were first introduced in the 1980s by Hopfield and Tank [20, 39]. Since then, neural networks have been applied to various optimization problems, including linear programming, nonlinear programming, variational inequali- ties, and linear and nonlinear complementarity problems; see [5, 9, 10, 11, 17, 22, 23, 25, 29, 41, 42, 43, 44, 45]. The main idea of the neural network approach for optimization is to construct a nonnegative energy function and establish a dynamic system that repre- sents an artificial neural network. The dynamic system is usually in the form of first order ordinary differential equations. Furthermore, it is expected that the dynamic system will approach its static state (or an equilibrium point), which corresponds to the solution for the underlying optimization problem, starting from an initial point. In addition, neu- ral networks for solving optimization problems are hardware-implementable; that is, the neural networks can be implemented by using integrated circuits. In this paper, we focus on neural network approach to the SOCCVI problem. Our neural networks will be aimed to solve the system (5) whose solutions are candidates of SOCCVI problem (1).
The rest of this paper is organized as follows. Section 2 introduces some preliminar- ies. In Section 3, the first neural network based on the Fischer-Burmeister function is proposed and studied. In Section 4, we show that the KKT system (5) is equivalent to a nonlinear projection formulation. Then, the model of neural network for solving the pro- jection formulation is introduced and its stability is analyzed. In Section 5, illustrative examples are discussed. Section 6 gives the conclusion of this paper.
2 Preliminaries
In this section, we recall some preliminary results that will be used later and background materials of ordinary differential equations that will play an important role in the sub- sequent analysis. We begin with some concepts for a nonlinear mapping.
Definition 2.1 Let F = (F1, . . . , Fn)T : Rn → Rn. Then, the mapping F is said to be (a) monotone if
hF (x) − F (y), x − yi ≥ 0, ∀x, y ∈ Rn. (b) strictly monotone if
hF (x) − F (y), x − yi > 0, ∀x, y ∈ Rn. (c) strongly monotone with constant η > 0 if
hF (x) − F (y), x − yi ≥ ηkx − yk2, ∀x, y ∈ Rn. (d) F is said to be Lipschitz continuous with constant γ if
kF (x) − F (y)k ≤ γkx − yk, ∀x, y ∈ Rn.
Definition 2.2 Let X be a closed convex set in Rn. Then, for each x ∈ Rn, there is a unique point y ∈ X such that kx − yk ≤ kx − zk, ∀z ∈ X. Here y is known as the projection of x onto the set X with respect to Euclidean norm, that is,
y = PX(x) = arg minz∈Xkx − zk.
The projection function PX(x) has the following property, called projection theorem [2], which is useful in our subsequent analysis.
Property 2.1 Let X be a nonempty closed convex subset of Rn. Then, for each z ∈ Rn, PX(z) is the unique vector ¯z ∈ X such that (y − ¯z)T(z − ¯z) ≤ 0, ∀y ∈ X.
Next, we recall some materials about first order differential equations (ODE):
˙
w(t) = H(w(t)), w(t0) = w0 ∈ Rn, (6) where H : Rn→ Rn is a mapping. We also introduce three kinds of stability that will be discussed later. These materials can be found in usual ODE textbooks, e.g. [32].
Definition 2.3 A point w∗ = w(t∗) is called an equilibrium point or a steady state of the dynamic system (6) if H(w∗) = 0. If there is a neighborhood Ω∗ ⊆ Rn of w∗ such that H(w∗) = 0 and H(w) 6= 0 ∀w ∈ Ω∗\ {w∗}, then w∗ is called an isolated equilibrium point.
Lemma 2.1 Assume that H : Rn → Rn is a continuous mapping. Then, for any t0 > 0 and w0 ∈ Rn, there exists a local solution w(t) for (6) with t ∈ [t0, τ ) for some τ > t0. If, in addition, H is locally Lipschitz continuous at w0, then the solution is unique; if H is Lipschitz continuous in Rn, then τ can be extended to ∞.
If a local solution defined on [t0, τ ) cannot be extended to a local solution on a larger interval [t0, τ1), τ1 > τ, then it is called a maximal solution, and the interval [t0, τ ) is the maximal interval of existence. Clearly, any local solution has an extension to a maximal one. We denote [t0, τ (w0)) by the maximal interval of existence associated with w0. Lemma 2.2 Assume that H : Rn → Rn is continuous. If w(t) with t ∈ [t0, τ (w0)) is a maximal solution and τ (w0) < ∞, then lim
t↑τ (w0)kw(t)k = ∞.
Definition 2.4 (Lyapunov Stability) Let w(t) be a solution for (6). An isolated equilib- rium point w∗ is Lyapunov stable if for any w0 = w(t0) and any ε > 0, there exists a δ > 0 such that if kw(t0) − w∗k < δ, then kw(t) − w∗k < ε for all t ≥ t0.
Definition 2.5 (Asymptotic Stability) An isolated equilibrium point w∗ is said to be asymptotic stable if in addition to being Lyapunov stable, it has the property that if kw(t0) − w∗k < δ, then w(t) → w∗ as t → ∞.
Definition 2.6 (Lyapunov function) Let Ω ⊆ Rn be an open neighborhood of ¯w. A continuously differentiable function V : Rn→ R is said to be a Lyapunov function at the state ¯w over the set Ω for equation (6) if
V ( ¯w) = 0, V (w) > 0, ∀w ∈ Ω \ { ¯w},
V (w) ≤ 0, ∀w ∈ Ω \ { ¯˙ w}. (7)
Lemma 2.3 (a) An isolated equilibrium point w∗ is Lyapunov stable if there exists a Lyapunov function over some neighborhood Ω∗ of w∗.
(b) An isolated equilibrium point w∗ is asymptotically stable if there exists a Lyapunov function over some neighborhood Ω∗ of w∗ such that ˙V (w) < 0, ∀w ∈ Ω∗\ {w∗}.
Definition 2.7 (Exponential Stability) An isolated equilibrium point w∗ is exponentially stable if there exists a δ > 0 such that arbitrary point w(t) of (6) with the initial condition w(t0) = w0 and kw(t0) − w∗k < δ is well defined on [t0, +∞) and satisfies
kw(t) − w∗k ≤ ce−ωtkw(t0) − w∗k ∀t ≥ t0, where c > 0 and ω > 0 are constants independent of the initial point.
3 Neural network model based on smoothed Fischer- Burmeister function
The smoothed Fischer-Burmeister function over the second-order cone defined below is used to construct a merit function by which the KKT system of SOCCVI is reformulated as an unconstrained smooth minimization problem. Furthermore, based on the mini- mization problem, we propose a neural network and study its stability in this section.
For any a = (a1; a2), b = (b1; b2) ∈ R × Rn−1, we define their Jordan product as a · b = (aTb; b1a2+ a1b2).
We denote a2 = a · a and |a| =√
a2, where for any b ∈ Kn, √
b is the unique vector in Kn such that b =√
b ·√ b.
Definition 3.1 A function φ : Rn×Rn→ Rnis called an SOC-complementarity function if it satisfies
φ(a, b) = 0 ⇐⇒ a · b = 0, a ∈ Kn, b ∈ Kn.
A popular SOC-complementarity function is the Fischer-Burmeister function, which is semismooth [34] and defined as
φFB(a, b) = a2 + b21/2
− (a + b).
Then the smoothed Fischer-Burmeister function is given by φεFB(a, b) = a2+ b2 + ε2e1/2
− (a + b) (8)
with ε ∈ R+ and e = (1, 0, · · · , 0)T ∈ Rn.
The following lemma gives the gradient of φεFB. Since the proofs can be found in [14, 34, 38], we here omit them.
Lemma 3.1 Let φεFB be defined as in (8) and ε 6= 0. Then, φεFB is continuously differen- tiable everywhere and
∇εφεFB(a, b) = eTL−1z Lεe, ∇aφεFB(a, b) = L−1z La− I, ∇bφεFB(a, b) = L−1z Lb− I, where z = (a2+ b2+ ε2e)1/2, I is identity mapping and La = a1 aT2
a2 a1In−1
for a = (a1; a2) ∈ R × Rn−1.
Using Definition 3.1 and KKT condition described in [38], we can see that the KKT system (5) is equivalent to the following unconstrained smooth minimization problem:
min Ψ(w) := 1
2kS(w)k2. (9)
Here Ψ(w), w = (ε, x, µ, λ) ∈ R1+n+l+m, is a merit function, and S(w) is defined by
S(w) =
ε L(x, µ, λ)
−h(x) φεFB(−gm1(x), λm1)
...
φεFB(−gmp(x), λmp)
,
with gmi(x), λmi ∈ Rmi. In other words, Ψ(w) given in (9) is a smooth merit function for the KKT system (5).
Based on the above smooth minimization problem (9), it is natural to propose the first neural network for solving the KKT system (5) of SOCCVI problem:
dw(t)
dt = −ρ∇Ψ(w(t)), w(t0) = w0, (10)
where ρ > 0 is a scaling factor.
Remark 3.1 In fact, we can also adopt another merit function which is based on the FB function without the element ε. That is, we can define
S(x, µ, λ) =
L(x, µ, λ)
−h(x) φFB(−gm1(x), λm1)
...
φFB(−gmp(x), λmp)
. (11)
Then, the neural network model (10) could be obtained as well because kφFBk2 is smooth [7]. However, it is observed that the gradient mapping ∇Ψ has more complicated formulas
because (−gmi(x))2 + λ2mi may lie on the boundary of SOC, or interior of SOC, see [7, 34], which will cost more expensive numerical computations. Thus, the one dimensional parameter ε in use not only has no influence on the main result, but also will simplify the computational work.
To discuss properties of the neural network model (10), we make the following as- sumption which is used to avoid the singularity of ∇S(w), see [38].
Assumption 3.1 (a) the gradients {∇hj(x)|j = 1, · · · , l} ∪ {∇gi(x)|i = 1, · · · , m} are linear independent.
(b) ∇xL(x, µ, λ) is positive definite on the null space of the gradients {∇hj(x)|j = 1, · · · , l}.
When SOCCVI problem corresponds to the KKT conditions of a convex second- order cone program (CSOCP) problem as (4) where both h and g are linear, the above Assumption 3.1(b) is indeed equivalent to the well-used condition ∇2f (x) is positive definite, e.g. [43, Corollary 1 ].
Proposition 3.1 Let Ψ : R1+n+l+m → R+ be defined as in (9). Then, Ψ(w) ≥ 0 for w = (ε, x, µ, λ) ∈ R1+n+l+m and Ψ(w) = 0 if and only if (x, µ, λ) solves the KKT system (5).
Proof. The proof is straightforward. 2
Proposition 3.2 Let Ψ : R1+n+l+m → R+ be defined as in (9). Then, the following results hold.
(a) The function Ψ is continuously differentiable everywhere with
∇Ψ(w) = ∇S(w)S(w), where
∇S(w) =
1 0 0 diag{∇εφεFB(−gmi(x), λmi)}pi=1
0 ∇xL(x, µ, λ)T −∇h(x) −∇g(x)diag{∇gmiφεFB(−gmi(x), λmi)}pi=1
0 ∇h(x)T 0 0
0 ∇g(x)T 0 diag{∇λmiφεFB(−gmi(x), λmi)}pi=1
.
(b) Suppose that assumption 3.1 holds. Then, ∇S(w) is nonsingular for any w ∈ R1+n+l+m. Moreover, if (0, x, µ, λ) ∈ R1+n+l+m is a stationary point of Ψ, then (x, µ, λ) ∈ Rn+l+m is a KKT triple of the SOCCVI problem.
(c) Ψ(w(t)) is nonincreasing with respect to t.
Proof. Part(a) follows from the chain rule. For part(b), we know that ∇S(w) is non- singular if and only if the following matrix
∇xL(x, µ, λ)T −∇h(x) −∇g(x)diag{∇gmiφεFB(−gmi(x), λmi)}pi=1
∇h(x)T 0 0
∇g(x)T 0 diag{∇λmiφεFB(−gmi(x), λmi)}pi=1
is nonsingular. In fact, from [38, Theorem 3.1] and [38, Proposition 4.1], the above matrix is nonsingular and (x, µ, λ) ∈ Rn+l+m is a KKT triple of the SOCCVI problem if (0, x, µ, λ) ∈ R1+n+l+m is a stationary point of Ψ. It remains to show part(c). By the definition of Ψ(w) and (10), it is not difficult to compute
dΨ(w(t))
dt = ∇Ψ(w(t))Tdw(t)
dt = −ρk∇Ψ(w(t))k2 ≤ 0. (12) Therefore, Ψ(w(t)) is a monotonically decreasing function with respect to t. 2
Now, we are ready to analyze the behavior of the solution trajectory of (10) and establish properties of three kinds of stability for an isolated equilibrium point.
Proposition 3.3 (a) If (x, µ, λ) ∈ Rn+l+m is a KKT triple of SOCCVI problem, then (0, x, µ, λ) ∈ R1+n+l+m is an equilibrium point of (10).
(b) If Assumption 3.1 holds and (0, x, µ, λ) ∈ R1+n+l+m is an equilibrium point of (10), then (x, µ, λ) ∈ Rn+l+m is a KKT triple of SOCCVI problem.
Proof. (a) From Proposition 3.1 and (x, µ, λ) ∈ Rn+l+m being a KKT triple of SOCCVI problem, it is clear that S(0, x, µ, λ) = 0. Hence, ∇Ψ(0, x, µ, λ) = 0. Besides, by Proposition 3.2, we know that if ε 6= 0, then ∇Ψ(ε, x, µ, λ) 6= 0. This shows that (0, x, µ, λ) is an equilibrium point of (10).
(b) It follows from (0, x, µ, λ) ∈ R1+n+l+m being an equilibrium point of (10) that
∇Ψ(0, x, µ, λ) = 0. In other words, (0, x, µ, λ) is the stationary point of Ψ. Then, the result is a direct consequence of Proposition 3.2(b). 2
Proposition 3.4 (a) For any initial state w0 = w(t0), there exists exactly one maximal solution w(t) with t ∈ [t0, τ (w0)) for the neural network (10).
(b) If the level set L(w0) = {w ∈ R1+n+l+m|Ψ(w) ≤ Ψ(w0)} is bounded, then τ(w0) = +∞.
Proof. (a) Since S is continuous differentiable, ∇S is continuous, and therefore, ∇S is bounded on a local compact neighborhood of w. That means ∇Ψ(w) is locally Lipschitz continuous. Thus, applying Lemma 2.1 leads to the desired result.
(b) This proof is similar to the proof of Case(i) in [5, Proposition 4.2]. 2
Remark 3.2 We wish to obtain the result that the level sets L(Ψ, γ) := {w ∈ R1+n+l+m | Ψ(w) ≤ γ}
are bounded for all γ ∈ R. However, we are not able to complete the argument. We suspect that there needs more subtle properties of F , h and g to finish it.
Next, we investigate the convergence of the solution trajectory of neural network (10)
Theorem 3.1 (a) Let w(t) with t ∈ [t0, τ (w0)) be the unique maximal solution to (10).
If τ (w0) = +∞ and {w(t)} is bounded, then limt→∞∇Ψ(w(t)) = 0.
(b) If Assumption 3.1 holds and (ε, x, µ, λ) ∈ R1+n+l+m is the accumulation point of the trajectory w(t), then (x, µ, λ) ∈ Rn+l+m is a KKT triple of SOCCVI problem.
Proof. With Proposition 3.2(b) and (c) and Proposition 3.4, the arguments are exactly the same as those for [29, Corollary 4.3]. Thus, we omit them. 2
Theorem 3.2 Let w∗ be an isolated equilibrium point of the neural network (10). Then the following results hold.
(a) w∗ is asymptotically stable.
(b) If Assumption 3.1 holds, then it is exponentially stable.
Proof. Since w∗ is an isolated equilibrium point of (10), there exists a neighborhood Ω∗ ⊆ R1+n+l+m of w∗ such that
∇Ψ(w∗) = 0 and ∇Ψ(w) 6= 0 ∀w ∈ Ω∗\ {w∗}.
Next, we argue that Ψ(w) is indeed a Lyapunov function at x∗ over the set Ω∗ for (10) by showing that the conditions in (7) are satisfied. First, notice that Ψ(w) ≥ 0.
Suppose that there is an ¯w ∈ Ω∗\ {w∗} such that Ψ( ¯w) = 0. Then, we can easily obtain that ∇Ψ( ¯w) = 0, i.e., ¯w is also an equilibrium point of (10), which clearly contradicts the assumption that w∗ is an isolated equilibrium point in Ω∗. Thus, we prove that
Ψ(w) > 0 for any w ∈ Ω∗\ {w∗}. This together with (12) shows that the condition in (7) are satisfied. Because w∗ is isolated, from (12), we have
dΨ(w(t))
dt < 0, ∀w(t) ∈ Ω∗\ {w∗}.
This implies that w∗ is asymptotically stable. Furthermore, if Assumption 3.1 holds, we can obtain that ∇S is nonsingular. In addition, we have
S(w) = S(w∗) + ∇S(w∗)(w − w∗) + o(kw − w∗k), ∀w ∈ Ω∗\ {w∗}. (13) From kS(w(t))k being a monotonically decreasing function with respect to t and (13), we can deduce that
kw(t) − w∗k ≤ k(∇S(w∗))−1kkS(w(t)) − S(w∗)k + o(kw(t) − w∗k)
≤ k(∇S(w∗))−1kkS(w(t0)) − S(w∗)k + o(kw(t) − w∗k)
≤ k(∇S(w∗))−1k [k(∇S(w∗))kkw(t0)−w∗k+o(kw(t0)−w∗k)]+o(kw(t) − w∗k).
That is,
kw(t) − w∗k−o(kw(t) − w∗k) ≤ k(∇S(w∗))−1k [k(∇S(w∗))kkw(t0)−w∗k+o(kw(t0)−w∗k)] . The above inequality implies that the neural network (10) is also exponentially stable.
2
4 Neural network model based on projection func- tion
In this section, we present that the KKT triple of SOCCVI problem is equivalent to the solution of a projection formulation. Based on this, we introduce another neural network model for solving the projection formulation and analyze the stability conditions and convergence.
Define the function U : Rn+l+m → Rn+l+m and vector w in the following form:
U(w) =
L(x, µ, λ)
−h(x)
−g(x)
, w =
x µ λ
, (14)
where L(x, µ, λ) = F (x)+∇h(x)µ+∇g(x)λ is the Lagrange function. To avoid confusion, we emphasize that, for any w ∈ Rn+l+m, we have
wi∈ R, if 1 ≤ i ≤ n + l,
wi∈ Rmi−(n+l), if n + l + 1 ≤ i ≤ n + l + p.
Then, we may write (14) as
Ui = (U(w))i = (L(x, µ, λ))i, wi = xi, i = 1, . . . , n, Un+j = (U(w))n+j = −hj(x), wn+j = µj, , j = 1, . . . , l,
Un+l+k = (U(w))n+l+k = −gk(x) ∈ Rmk, wn+l+k = λk∈ Rmk, k = 1, . . . , p,
p
X
k=1
mk = m.
With this, the KKT conditions (5) can be recast as Ui = 0, i = 1, 2, . . . , n, n + 1, . . . , n + l,
hUJ, wJi = 0, UJ = (Un+l+1, Un+l+2, · · · , Un+l+p)T ∈ K, wJ = (wn+l+1, wn+l+2, · · · , wn+l+p)T ∈ K.
(15)
Thus, (x∗, µ∗, λ∗) is a KKT triple for (1) if and only if (x∗, µ∗, λ∗) is a solution to (15).
It is well known that the nonlinear complementarity problem, which is denoted by NCP(F, K) and to find an x ∈ Rn such that
x ∈ K, F (x) ∈ K and hF (x), xi = 0
where K is a closed convex set of Rn, is equivalent to the following VI(F, K) problem:
finding an x ∈ K such that
hF (x), y − xi ≥ 0 ∀y ∈ K.
Furthermore, if K = Rn, then NCP(F, K) becomes the system of nonlinear equations F (x) = 0.
Based on the above, solution of (15) is equivalent to solution of the following VI problem: find w ∈ K such that
hU(w), v − wi ≥ 0, ∀v ∈ K, (16)
where K = Rn+l× K. In addition, by applying the Property 2.1, its solution is equivalent to solution of below projection formulation
PK(w − U(w)) = w with K = Rn+l × K, (17) where function U and vector w are defined in (14). Now, according to (17), we give the following neural network:
dw
dt = ρ{PK(w − U(w)) − w}, (18)
where ρ > 0. Note that K is a closed and convex set. For any w ∈ Rn+l+m, PK means PK(w) = [PK(w1), PK(w2), · · · , PK(wn+l), PK(wn+l+1), PK(wn+l+2), · · · , PK(wn+l+p)],
where
PK(wi) = wi, i = 1, · · · , n + l,
PK(wn+l+j) = [λ1(wn+l+j)]+· u(1)wn+l+j+ [λ2(wn+l+j)]+· u(2)wn+l+j, j = 1, · · · , p.
Here, for the sake of simplicity, we denote the vector wn+l+j by v for the moment, and [·]+ is the scalar projection, λ1(v), λ2(v) and u(1)v , u(2)v are the spectral values and the associated spectral vectors of v = (v1; v2) ∈ R × Rmj−1, respectively, given by
( λi(v) = v1+ (−1)ikv2k, u(i)v = 12
1, (−1)i vkv22k , for i = 1, 2, see [7, 34].
The dynamic system described by (18) can be recognized as a recurrent neural network with a single-layer structure. To analyze the stability conditions of (18), we need the following lemmas and proposition.
Lemma 4.1 If the gradient of L(x, µ, λ) is positive semi-definite (respectively, positive definite), then the gradient of U in (14) is positive semi-definite (respectively, positive definite).
Proof. Since we have
∇U(x, µ, λ) =
∇xLT(x, µ, λ) −∇h(x) −∇g(x)
∇Th(x) 0 0
∇Tg(x) 0 0
,
for any nonzero vector d = (pT, qT, rT)T ∈ Rn+l+m, we can obtain that
dT∇U(x, µ, λ)d = pT qT rT
∇xLT(x, µ, λ) −∇h(x) −∇g(x)
∇Th(x) 0 0
∇Tg(x) 0 0
p q r
= pT∇xL(x, µ, λ)p.
This leads to the desired results. 2
Proposition 4.1 For any initial point w0 = (x0, µ0, λ0) with λ0 := λ(t0) ∈ K, there exist a unique solution w(t) = (x(t), µ(t), λ(t)) for neural network (18), Moreover, λ(t) ∈ K.
Proof. For simplicity, we assume K = Km. The analysis can be carried over to the general case where K is the Cartesian product of second-order cones. Since F, h, g are continuous differentiable, the function
F (w) := PK(w − U(w)) − w with K = Rn+l× Km (19) is semi-smooth and Lipschitz continuous. Thus, there exists a unique solution w(t) = (x(t), µ(t), λ(t)) for neural network (18). Therefore, it remains to show that λ(t) ∈ Km. For convenience, we denote λ(t) := (λ1(t), λ2(t)) ∈ R × Rm−1. To complete the proof, we need to verify two things: (i) λ1(t) ≥ 0 and (ii) kλ2(t)k ≤ λ1(t). First, from (18), we
have dλ
dt + ρλ(t) = ρPKm(λ + g(x)).
The solution of the above first-order ordinary differential equation is λ(t) = e−ρ(t−t0)λ(t0) + ρe−ρt
Z t t0
ρeρsPKm(λ + g(x))ds. (20)
If we let λ(t0) := (λ1(t0), λ2(t0)) ∈ R × Rm−1 and denote PKm(λ + g(x)) as z(t0) :=
(z1(t0), z2(t0)), then (20) leads to
λ1(t) = e−ρ(t−t0)λ1(t0) + ρe−ρt Z t
t0
ρeρsz1(s)ds, (21)
λ2(t) = e−ρ(t−t0)λ2(t0) + ρe−ρt Z t
t0
ρeρsz2(s)ds. (22) Due to both λ(t0) and z(t) belong to Km, there have λ1(t0) ≥ 0, kλ2(t0)k ≤ λ1(t0) and kz2(t)k ≤ z1(t). Therefore, λ1(t) ≥ 0 since both terms in the right-hand side of (21) are nonnegative. In addition,
kλ2(t)k ≤ e−ρ(t−t0)kλ2(t0)k + ρe−ρtRt
t0ρeρskz2(s)kds
≤ e−ρ(t−t0)λ1(t0) + ρe−ρtRt
t0ρeρsz1(s)ds
= λ1(t), which implies that λ(t) ∈ Km 2
Lemma 4.2 Let U(w), F (w) be defined as in (14) and (19), respectively. Suppose w∗ = (x∗, µ∗, λ∗) is an equilibrium point of neural network (18) with (x∗, µ∗, λ∗) being an KKT triple of SOCCVI problem. Then, the following inequality holds:
(F (w) + w − w∗)T(−F (w) − U(w)) ≥ 0. (23)
Proof. Notice that
(F (w) + w − w∗)T(−F (w) − U(w))
= [−w + PK(w − U(w)) + w − w∗]T[w − PK(w − U(w)) − U(w)]
= [−w∗+ PK(w − U(w))]T[w − PK(w − U(w)) − U(w)]
= −[w∗− PK(w − U(w))]T[w − U(w) − PK(w − U(w))].
Since w∗ ∈ K, applying Property 2.1 gives
[w∗− PK(w − U(w))]T[w − U(w) − PK(w − U(w))] ≤ 0.
Thus, we have
(F (w) + w − w∗)T(−F (w) − U(w)) ≥ 0.
This completes the proof. 2
We now show the stability and convergence issues regarding neural network (18).
Theorem 4.1 If ∇xL(w) is positive semi-definite (respectively, positive definite), the the solution of neural network (18) with initial point w0 = (x0, µ0, λ0) where λ0 ∈ K is Lyapunov stable (respectively, asymptotically stable). Moreover, the solution trajectory of neural network (18) is extendable to the global existence.
Proof. Again, for simplicity, we assume K = Km. From Proposition 4.1, there exists a unique solution w(t) = (x(t), µ(t), λ(t)) for neural network (18) and λ(t) ∈ Km. Let w∗ = (x∗, µ∗, λ∗) be an equilibrium point of neural network (18). We define a Lyapunov function as below:
V (w) := V (x, µ, λ) := −U(w)TF (w) − 1
2kF (w)k2+ 1
2kw − w∗k2. (24) From [12, Theorem 3.2], we know that V is continuously differentiable with
∇V (w) = U(w) − [∇U(w) − I]F (w) + (w − w∗).
It is also trivial that V (w∗) = 0. Then, we have dV (w(t))
dt = ∇V (w(t))Tdw
= {U(w) − [∇U(w) − I]F (w) + (w − wdt ∗)}TρF (w)
= ρ{[U(w) + (w − w∗)]TF (w) + kF (w)k2− F (w)T∇U(w)F (w)}.
Inequality (23) in Lemma 4.2 implies
[U(w) + (w − w∗)]TF (w) ≤ −U(w)T(w − w∗) − kF (w)k2,
which yields dV (w(t))
≤ ρ{−U(w)dt T(w − w∗) − F (w)T∇U(w)F (w)}
= ρ{−U(w∗)T(w − w∗) − (U(w) − U(w∗))T(w − w∗) − F (w)T∇U(w)F (w)}.
(25)
Note that w∗ is the solution of the variational inequality (16). Since w ∈ K, we therefore obtain −U(w∗)T(w − w∗) ≤ 0. Because U(w) is continuous differentiable and ∇U(w) is positive semi-definite, by [33, Theorem 5.4.3], we obtain that U(w) is monotone. Hence, we have −(U(w) − U(w∗))T(w − w∗) ≤ 0 and −F (w)T∇U(w)F (w) ≤ 0. The above discussions lead to dV (w(t))dt ≤ 0. Also, by [33, Theorem 5.4.3], we know that if ∇U(w) is positive definite, then U(w) is strictly monotone, which implies dV (w(t))dt < 0 in this case.
In order to obtain V (w) is a Lyapunov function and w∗ is Lyapunov stable, we will show the following inequality:
−U(w)TF (w) ≥ kF (w)k2. (26)
To see this, we first observe that
kF (w)k2+ U(w)TF (w) = [w − PK(w − U(w))]T[w − U(w) − PK(w − U(w))].
Since w ∈ K, applying Property 2.1 again, there holds
[w − PK(w − U(w))]T[w − U(w) − PK(w − U(w))] ≤ 0,
which yields the desired inequality (26). By combining equation (24) and (26), we have V (w) ≥ 1
2kF (w)k2+ 1
2kw − w∗k2,
which says V (w) > 0 if w 6= w∗. Hence V (w) is indeed a Lyapunov function and w∗ is Lyapunov stable. Furthermore, if ∇xL(w) is positive definite, we have w∗ is asymptoti- cally stable. Moreover, it holds that
V (w0) ≥ V (w) ≥ 1
2kw − w∗k2 for t ≥ t0, (27) which tells us the solution trajectory w(t) is bounded. Hence, it can be extended to global existence. 2
Theorem 4.2 Let w∗ = (x∗, µ∗, λ∗) be an equilibrium point of (18). If ∇xL(w) is positive definite, the solution of neural network (18) with initial point w0 = (x0, µ0, λ0) where λ0 ∈ K is globally convergent to w∗ and has finite convergence time.
Proof. From (27), the level set
L(w0) := {w|V (w) ≤ V (w0)}
is bounded. Then, the Invariant Set Theorem [15] implies the solution trajectory w(t) converges to θ as t → +∞ where θ is the largest invariant set in
Λ =
w ∈ L(w0)
dV (w(t)) dt = 0
.
We will show that dw/dt = 0 if and only if dV (w(t))/dt = 0 which yields that w(t) converges globally to the equilibrium point w∗ = (x∗, µ∗, λ∗). Suppose dw/dt = 0, then it is clear that dV (w(t))/dt = ∇V (w)T(dw/dt) = 0. Let ˆw = (ˆx, ˆµ, ˆλ) ∈ Λ which says dV ( ˆw(t))/dt = 0. From (23), we know that
dV ( ˆw(t))/dt ≤ ρ{(−U( ˆw) − U(w∗))T( ˆw − w∗) − F ( ˆw)T∇U( ˆw)F ( ˆw)}.
Both terms inside the big parenthesis are nonpositive as shown in Theorem 4.1, so (U( ˆw)−
U(w∗))T( ˆw − w∗) = 0, F ( ˆw)T∇U( ˆw)F ( ˆw) = 0. The condition ∇xL(w) being positive definite leads to ∇U( ˆw) being positive definite. Hence,
F ( ˆw) = − ˆw + PK( ˆw − U( ˆw)) = 0,
which is equivalent to d ˆw/dt = 0. From the above, w(t) converges globally to the equilibrium point w∗= (x∗, µ∗, λ∗). Moreover, with Theorem 4.1 and following the same argument as in [43, Theorem 2], the neural network (18) has finite convergence time.
2
5 Simulations
To demonstrate effectiveness of the proposed neural networks, some illustrative SOCCVI problems are tested. The numerical implementation is coded by Matlab 7.0 and the ordinary differential equation solver adopted is ode23, which uses Runge-Kutta (2; 3) formula. In the following tests, the parameter ρ in both neural networks is set to be 1000.
Example 5.1 Consider the SOCCVI problem (1)-(2) where
F (x) =
2x1+ x2+ 1 x1+ 6x2− x3− 2
−x2+ 3x3−65x4+ 3
−65x3+ 2x4+ 12sin x4cos x5sin x6+ 6
1
2cos x4sin x5sin x6+ 2x5−52
−12 cos x4cos x5cos x6+ 2x6 +14cos x6sin x7cos x8+ 1
1
4sin x6cos x7cos x8+ 4x7− 2
−14sin x6sin x7sin x8+ 2x8+12
and
C = {x ∈ R8 : −g(x) = x ∈ K3× K3× K2}.
This problem has an approximate solution
x∗ = (0.3820, 0.1148, −0.3644, 0.0000, 0.0000, 0.0000, 0.5000, −0.2500)T. It can be verified that the Lagrangian function for this example is
L(x, µ, λ) = F (x) − λ and the gradient of the Lagrangian function is
∇L(x, µ, λ) = ∇F (x) I8×8
,
where I is the identity mapping and ∇F (x) means the gradient of F (x). We use the proposed neural networks with smoothed FB and projection functions, respectively, to solve the problem whose trajectories are depicted in Figures 1 and 2. The simulation results show that both trajectories are globally convergent to x∗ and the neural network with projection function converges to x∗ quicker than that with smoothed FB function.
0 10 20 30 40 50 60 70 80
−1.5
−1
−0.5 0 0.5 1
Time (ms)
Trajectories of x(t)
x1 x2
x3
x4 x5
x6
x8 x7
Figure 1: Transient behavior of neural network with smoothed FB function in Example 5.1.
0 5 10 15 20 25
−3
−2.5
−2
−1.5
−1
−0.5 0 0.5 1 1.5
Time (ms)
Trajectories of x(t)
x1 x2
x3
x4 x5
x6
x8 x7
Figure 2: Transient behavior of the neural network with projection function in Example 5.1.
Example 5.2 [38, Example 5.1] We consider the following SOCCVI problem:
1
2Dx, y − x
≥ 0, ∀y ∈ C where
C = {x ∈ Rn: Ax − a = 0, Bx − b 0},
D is an n × n symmetric matrix, A and B are l × n and m × n matrices, respectively, d is an n × 1 vector, a and b are l × 1 and m × 1 vectors with l + m ≤ n, respectively.
In fact, we can determine the data a, b, A, B and D randomly. However, as in [38, Example 5.1], we set the data as follows:
D = [Dij]n×n, where Dij =
2, i = j 1, |i − j| = 1 0, otherwise
,
A = [Il×l 0l×(n−l)]l×n, B = [0m×(n−m) Im×m]m×n, a = 0l×1, b = (em1, em2, · · · , emp)T, where emi = (1, 0, · · · , 0)T ∈ Rmi and l + m ≤ n. Clearly, A and B are full row rank and rank([AT BT]) = l + m.
In the simulation, the parameters l, m, and n are set to be 3,3, and 6, respectively. The problem has an solution x∗ = (0, 0, 0, 0, 0, 0)T. It can be verified that the Lagrangian function for this example is
L(x, µ, λ) = 1
2Dx + ATµ + BTλ.
Note that ∇xL(x, µ, λ) is positive definite. We know from Theorems 3.1 and 4.2 that both proposed neural networks are globally convergent to the KKT triple of the SOC- CVI problem. Figures 3 and 4 depict the trajectories of Example 5.2 obtained using the proposed neural networks. The simulation results show that both neural networks are effective in the SOCCVI problem and the neural network with projection function converges to x∗ quicker than that with smoothed FB function.
0 20 40 60 80 100 120 140 160
−0.2
−0.15
−0.1
−0.05 0 0.05 0.1 0.15 0.2 0.25
Time (ms)
Trajectories of x(t)
x5
x1 x3
x2
x4 x6
Figure 3: Transient behavior of neural network with smoothed FB function in Example 5.2.
Example 5.3 Consider the SOCCVI problem (1)-(2) where
F (x) =
x3exp(x1x3) + 6(x1+ x2) 6(x1+ x2) + 2(2x2− x3)
p1 + (2x2− x3)2 x1exp(x1x3) − 2x2− x3
p1 + (2x2− x3)2 x4
x5
0 5 10 15 20 25 30
−0.4
−0.3
−0.2
−0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
Time (ms)
Trajectories of x(t)
x6 x4 x1 x3 x5
x2
Figure 4: Transient behavior of neural network with projection function in Example 5.2.
and
C = {x ∈ R5 : h(x) = 0, −g(x) ∈ K3× K2}, with
h(x) = −62x31+ 58x2+ 167x33− 29x3− x4− 3x5+ 11,
g(x) =
−3x31− 2x2+ x3− 5x33
5x31− 4x2+ 2x3− 10x33
−x3
−x4
−3x5
.
This problem has an approximate solution x∗ = (0.6287, 0.0039, −0.2717, 0.1761, 0.0587)T. Example 5.4 Consider the SOCCVI problem (1)-(2) where
F (x) =
4x1 − sin x1cos x2+ 1
− cos x1sin x2+ 6x2+95x3+ 2
9
5x2+ 8x3+ 3 2x4+ 1
and
C =
x ∈ R4 : h(x) =x21−101x2x3+ x3
x23+ x4
= 0, −g(x) =x1 x2
∈ K2
.
This problem has an approximate solution x∗ = (0.2391, −0.2391, −0.0558, −0.0031)T. The neural network (10) based on smoothed FB function can solve Examples 5.3-5.4 sucessfully, see Figures 5-6, whereas the neural network (18) based on projection function fails to solve them. This is because that ∇xL(x, µ, λ) is not always positive definite in Examples 5.3 and 5.4. Hence, the neural network with projection function is not effective in these two problems. To the contrast, though there is no guarantee that the Assumption 3.1 (b) holds, the neural network with smoothed FB function is asymptotically stable from Theorem 3.2. Figures 5 and 6 depict the trajectories obtained using the neural network with the smoothed FB function for Examples 5.3 and 5.4, respectively. The simulation results show that each trajectory converges to the desired isolated equilibrium point which is exactly the approximate solution of Example 5.3 and Example 5.4, respectively.
0 10 20 30 40 50 60
−0.4
−0.2 0 0.2 0.4 0.6 0.8 1
Time (ms)
Trajectories of x(t)
x1
x2
x3 x4 x5
Figure 5: Transient behavior of neural network with smoothed FB function in Example 5.3.
Example 5.5 Consider the nonlinear convex SOCP [24] given by min exp(x1− x3) + 3(2x1− x2)4+p1 + (3x2 + 5x3)2
s.t. − g(x) =
4x1+ 6x2+ 3x3− 1
−x1+ 7x2− 5x3 + 2 x1
x2
x3
∈ K2 × K3
0 50 100 150
−0.5
−0.4
−0.3
−0.2
−0.1 0 0.1 0.2 0.3 0.4 0.5
Time (ms)
Trajectories of x(t)
x1
x2 x3 x4
Figure 6: Transient behavior of neural network with smoothed FB function in Example 5.4.
The approximate solution of this problem is x∗ = (0.2324, −0.07309, 0.2206)T. As men- tioned in Section 1, the CSOCP in Example 5.5 can be transformed into an equivalent SOCCVI problem. There are neural network models proposed for CSOCP in [28]. We here try a different approach for it. In other words, we use the proposed neural net- works with the smoothed FB and projection functions, respectively, to solve the problem whose trajectories are depicted in Figures 7 and 8. From the simulation results, we see that the neural network with smoothed FB function converges very slowly and it is not clear whether it converges to the solution in finite time or not. In view of these, to solve CSOCP, it seems better to apply the models introduced in [28] directly instead of transforming CSOCP into SOCCVI problem.
The simulation results of Examples 5.1, 5.2 and 5.5 tell us that the neural network with projection function converges to x∗ quicker than that with the smoothed FB function.
In general, the neural network with projection function has lower model complexity than that with the smoothed FB function. Hence, the neural network with projection function is preferable to the neural network with the smoothed FB function when both can globally converge to the solution of SOCCVI problem. On the other hand, from Examples 5.3 and 5.4, the neural network with smoothed FB function seems better for use when the positive semidefinite condition of ∇xL(x, µ, λ) is not satisfied.
0 50 100 150 200
−0.3
−0.2
−0.1 0 0.1 0.2 0.3 0.4 0.5
Time (ms)
Trajectories of x(t)
x1
x2 x3
Figure 7: Transient behavior of neural network with smoothed FB function in Example 5.5.
0 20 40 60 80 100 120
−0.2
−0.1 0 0.1 0.2 0.3 0.4 0.5
Time (ms)
Trajectories of x(t)
x1
x2 x3
Figure 8: Transient behavior of neural network with projection function in Example 5.5.
6 Conclusions
In this paper, We use the proposed neural networks with smoothed Fischer-Burmeister and projection functions to solve the SOCCVI problems. The first neural network uses the Fischer-Burmeister (FB) function to achieve an unconstrained minimization which is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit func- tion is a Lyapunov function and this neural network is asymptotically stable. Under Assumption 3.1, we prove that if (ε, x, µ, λ) ∈ R1+n+l+m is the accumulation point of the trajectory, then (x, µ, λ) ∈ Rn+l+m is a KKT triple of SOCCVI problem and the neural network is exponentially stable. The second neural network is introduced for solving a projection formulation whose solutions coincide with the KKT triples of SOCCVI prob- lem under the positive semidefinite condition of ∇xL(x, µ, λ). Its Lyapunov stability and global convergence are proved. Simulations show that both neural networks have merits of their own.
Acknowledgements. The authors thank the referees for their carefully reading this paper and helpful suggestions.
References
[1] M. Aghassi, D. Bertsimas, and G. Perakis, Solving asymmetric variational inequalities via convex optimization, Operations Research Letters, 34(2006), 481–490.
[2] D. P. Bertsekas, Nonlinear programming, Athena Scientific, 1995.
[3] J.-S. Chen, The semismooth-related properties of a merit function and a decent method for the nonlinear complementarity problem, Journal of Global Optimization, 36(2006), 565–580.
[4] J.-S. Chen, H.-T. Gao, and S.-H. Pan, An R-linearly convergent derivative-free algorithm for nonlinear complementarity problems based on the generalized Fischer- Burmeister merit function, Journal of Computational and Applied Mathematics, 232(2009), 455–471.
[5] J.-S. Chen, C.-H. Ko, and S.-H. Pan, A neural network based on the general- ized Fischer-Burmeister function for nonlinear complementarity problems, Informa- tion Sciences, vol. 180(2010), 697–711.
[6] J.-S. Chen and S.-H. Pan, A family of NCP functions and a descent method for the nonlinear complementarity problem, Computational Optimization and Applications, 40(2008), 389–404.
[7] J.-S. Chen and P. Tseng, An unconstrained smooth minimization reformula- tion of the second-order cone complementarity problem, Mathematical Programming, 104(2005), 293–327.
[8] X. Chen, L. Qi, and D. Sun, Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities, Mathematics of Computation, 67(222)(1998), 519–540.
[9] C. Dang, Y. Leung, X. Gao, and K. Chen, Neural networks for nonlinear and mixed complementarity problems and their applications, Neural Networks, 17(2004), 271–283.
[10] S. Effati, A. Ghomashi, and A. R. Nazemi, Applocation of projection neural network in solving convex programming problems, Applied Mathematics and Compu- tation, 188(2007), 1103–1114.
[11] S. Effati and A. R. Nazemi, Neural network and its application for solving linear and quadratic programming problems, Applied Mathematics and Computation, 172(2006), 305–331.
[12] M. Fukushima, Equivalent differentiable optimization problems and descent meth- ods for asymmetric variational inequality problems, Mathematical Programming, 53(1)(1992), 99–110.
[13] F. Facchinei, J. Pang, Finite-dimensional Variational Inequalities and Comple- mentarity Problems, Springer, New York, 2003.
[14] M. Fukushima, Z.-Q. Luo, and P. Tseng, Smoothing functions for second- order-cone complimentarity problems, SIAM Journal on Optimization, 12(2002), 436–
460.
[15] R. Golden, Mathematical methods for neural network analysis and design, The MIT Press, 1996.
[16] J. Hammond, Solving asymmetric variational inequality problems and systems of equations with generalized nonlinear programming algorithms, Ph.D. Dissertation, De- partment of Mathematics, MIT, Cambridge, 1984.
[17] Q. Han, L.-Z. Liao, H. Qi, and L. Qi, Stability analysis of gradient-based neural networks for optimiation problems, Journal of Global Optimization, 19(2001), 363–
381.
[18] P. Harker and J.-S. Pang, Finite-dimensional variational inequality and non- linear complementarity problems: a survey of theory, algorithms and applications, Mathematical Programming, 48(1990), 161–220.
[19] P. Hartman and G. Stampacchia, On some nonlinear elliptic differential func- tional equations, Acta Mathematica, 115(1966), 271–310.
[20] J. J. Hopfield and D. W. Tank, Neural computation of decision in optimization problems, Biological Cybernetics, 52(1985), 141–152.
[21] X. Hu and J. Wang, A recurrent neural network for solving nonlinear con- vex programs subject to linear constraints, IEEE Transactions on Neural net- work,16(3)(2005), 379–386.
[22] X. Hu and J. Wang, Solving pseudomonotone variational inequalities and pseudo- convex optimization problems using the projection neural network, IEEE Transactions on Neural Network, 17(2006), 1487–1499.
[23] X. Hu and J. Wang, A recurrent neural network for solving a class of general variational inequalities, IEEE Transactions on Systems, Man, and Cybernetics-B, 37(2007), 528–539.
[24] C. Kanzow, I. Ferenczi, and M. Fukushima, On the local convergence of semismooth Newton methods for linear and nonlinear second-order cone programs without strict complementarity, SIAM Journal on Optimization, 20(2009), 297–320.
[25] M. P. Kennedy and L. O. Chua, Neural network for nonlinear programming, IEEE Transactions on Circuits and Systems, 35(1988), 554–562.
[26] Y. Kanno, J. A. C. Martins, and A. Pinto da Costa, Three-dimensional quasi-static frictional contact by using second-order cone linear complementarity prob- lem, International Journal for Numerical Methods in Engneering, 65(2006), 62–83.
[27] D. Kinderlehrer and G. Stampacchia, An introduction to variational inequal- ities and their applications, Academic Press, New York, 1980.
[28] C.-H. Ko, J.-S. Chen, and C.-Y. Yang, A recurrent neural network for solving nonlinear second-order cone programs, submitted manuscript, 2009.
[29] L.-Z. Liao, H. Qi, and L. Qi, Solving nonlinear complementarity problems with neural networks: a reformulation method approach, Jorunal of Computational and Applied Mathematics, 131(2001), 342–359.
[30] J. Lions and G. Stampacchia, Variational inequalities, Communications on Pure and Applied Mathematics, 20(1967), 493–519.
[31] O. Mancino and G. Stampacchia, Convex programming and variational inequal- ities, Journal of Optimization Theory and Applications, 9(1972), 3–23.
[32] R. K. Miller and A. N. Michel, Ordinary differential equations, Academic Press, 1982.
