to appear in Computational Optimization and Applications, 2010

### Neural networks for solving second-order cone constrained variational inequality problem

Juhe Sun ^{1}
School of Science

Shenyang Aerospace University Shenyang 110136, China

E-mail: juhesun@abel.math.ntnu.edu.tw

Jein-Shan Chen ^{2}
Department of Mathematics
National Taiwan Normal University

Taipei 11677, Taiwan E-mail: jschen@math.ntnu.edu.tw

Chun-Hsu Ko

Department of Electrical Engineering I-Shou University

Kaohsiung 840, Taiwan E-mail: chko@isu.edu.tw

April 18, 2010 (revised on July 12, 2010)

Abstract. In this paper, we consider using the neural networks to efficiently solve the second-order cone constrained variational inequality (SOCCVI) problem. More specifi- cally, two kinds of neural networks are proposed to deal with the Karush-Kuhn-Tucker (KKT) conditions of the SOCCVI problem. The first neural network uses the Fischer- Burmeister (FB) function to achieve an unconstrained minimization which is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit function is a Lyapunov function and this neural network is asymptotically stable. The second neural network is introduced for solving a projection formulation whose solutions coincide with the KKT triples of SOCCVI problem. Its Lyapunov stability and global convergence

1also affiliated with Department of Mathematics, National Taiwan Normal University.

2Member of Mathematics Division, National Center for Theoretical Sciences, Taipei Office. The author’s work is partially supported by National Science Council of Taiwan.

are proved under some conditions. Simulations are provided to show effectiveness of the proposed neural networks.

Key words. Second-order cone, variational inequality, Fischer-Burmeister function, neu- ral network, Lyapunov stable, projection function.

### 1 Introduction

Variational inequality (VI) problem, which was introduced by Stampacchia and his col- laborators [19, 30, 31, 36, 37], has attracted much attention from researchers of engi- neering, mathematics, optimization, transportation science, and economics communities, see [1, 26, 27]. It is well known that VIs subsume many other mathematical problems, including the solution of systems of equations, complementarity problems, and a class of fixed point problems. For a complete discussion and history of the finite VI problem and its associated solution methods, we refer the interested readers to the excellent survey text by Facchinei and Pang [13], the monograph by Patriksson [35], the survey article by Harker and Pang [18], the Ph.D. thesis of Hammond [16] and the references therein.

In this paper, we are interested in solving the second-order cone constrained varia- tional inequality (SOCCVI) problem whose constraints involve the Cartesian product of second-order cones (SOCs). The problem is to find x ∈ C satisfying

hF (x), y − xi ≥ 0, ∀y ∈ C, (1)

where the set C is finitely representable as

C = {x ∈ R^{n}: h(x) = 0, −g(x) ∈ K}. (2)
Here h·, ·i denotes the Euclidean inner product, F : R^{n}→ R^{n}, h : R^{n}→ R^{l} and g : R^{n}→
R^{m} are continuously differentiable functions and K is a Cartesian product of second-order
cones (or Lorentz cones), expressed as

K = K^{m}^{1} × K^{m}^{2} × · · · × K^{m}^{p}, (3)
where l ≥ 0, m1, m_{2}, · · · , mp ≥ 1, m1+ m_{2} + · · · + mp = m, and

K^{m}^{i} := {(x^{i1}, xi2, · · · , x^{im}i)^{T} ∈ R^{m}^{i} | k(x^{i2}, · · · , x^{im}i)k ≤ x^{i1}}

with k · k denoting the Euclidean norm and K^{1} the set of nonnegative reals R+. A special
case of equation (3) is K = R^{n}+, namely the nonnegative orthant in R^{n}, which corresponds
to p = n and m_{1} = · · · = mp = 1. When h is affine, an important special case of the
SOCCVI problem corresponds to the KKT conditions of the convex second-order cone
program (CSOCP):

min f (x)

s.t. Ax = b, −g(x) ∈ K, (4)

where A ∈ R^{l×n} has full row rank, b ∈ R^{l}, g : R^{n} → R^{m} and f : R^{n} → R. Furthermore,
when f is a convex twice continuously differentiable function, problem (4) is equivalent
to the following SOCCVI problem: Find x ∈ C satisfying

h∇f(x), y − xi ≥ 0, ∀y ∈ C, where

C = {x ∈ R^{n}: Ax − b = 0, −g(x) ∈ K}.

For solving the constrained variational inequalities and complementary problems (CP), many computational methods have been proposed, see [3, 4, 6, 8, 13, 40] and references therein. These methods include the method based on merit function, interior method, Newton method, nonlinear equation method, projection method and its variant versions. Another class of techniques for solving the VI problem exploits the fact that the KKT conditions of a VI problem comprise a mixed complementarity problem (MiCP), involving both equations and nonnegativity constraints. In other words, the SOCCVI problem can be solved by analyzing its KKT conditions:

L(x, µ, λ) = 0,

hg(x), λi = 0, −g(x) ∈ K, λ ∈ K, h(x) = 0,

(5)

where L(x, µ, λ) = F (x) + ∇h(x)µ + ∇g(x)λ is the variational inequality Lagrangian
function, µ ∈ R^{l} and λ ∈ R^{m}. However, in many scientific and engineering applications,
it is desirable to have a real-time solution for the VI and CP problems. Thus, at present,
for solving the VI and CP problems, many researchers employ the neural network method
which is a promising way to overcome this problem.

Neural networks for optimization were first introduced in the 1980s by Hopfield and Tank [20, 39]. Since then, neural networks have been applied to various optimization problems, including linear programming, nonlinear programming, variational inequali- ties, and linear and nonlinear complementarity problems; see [5, 9, 10, 11, 17, 22, 23, 25, 29, 41, 42, 43, 44, 45]. The main idea of the neural network approach for optimization is to construct a nonnegative energy function and establish a dynamic system that repre- sents an artificial neural network. The dynamic system is usually in the form of first order ordinary differential equations. Furthermore, it is expected that the dynamic system will approach its static state (or an equilibrium point), which corresponds to the solution for the underlying optimization problem, starting from an initial point. In addition, neu- ral networks for solving optimization problems are hardware-implementable; that is, the neural networks can be implemented by using integrated circuits. In this paper, we focus on neural network approach to the SOCCVI problem. Our neural networks will be aimed to solve the system (5) whose solutions are candidates of SOCCVI problem (1).

The rest of this paper is organized as follows. Section 2 introduces some preliminar- ies. In Section 3, the first neural network based on the Fischer-Burmeister function is proposed and studied. In Section 4, we show that the KKT system (5) is equivalent to a nonlinear projection formulation. Then, the model of neural network for solving the pro- jection formulation is introduced and its stability is analyzed. In Section 5, illustrative examples are discussed. Section 6 gives the conclusion of this paper.

### 2 Preliminaries

In this section, we recall some preliminary results that will be used later and background materials of ordinary differential equations that will play an important role in the sub- sequent analysis. We begin with some concepts for a nonlinear mapping.

Definition 2.1 Let F = (F1, . . . , Fn)^{T} : R^{n} → R^{n}. Then, the mapping F is said to be
(a) monotone if

hF (x) − F (y), x − yi ≥ 0, ∀x, y ∈ R^{n}.
(b) strictly monotone if

hF (x) − F (y), x − yi > 0, ∀x, y ∈ R^{n}.
(c) strongly monotone with constant η > 0 if

hF (x) − F (y), x − yi ≥ ηkx − yk^{2}, ∀x, y ∈ R^{n}.
(d) F is said to be Lipschitz continuous with constant γ if

kF (x) − F (y)k ≤ γkx − yk, ∀x, y ∈ R^{n}.

Definition 2.2 Let X be a closed convex set in R^{n}. Then, for each x ∈ R^{n}, there is
a unique point y ∈ X such that kx − yk ≤ kx − zk, ∀z ∈ X. Here y is known as the
projection of x onto the set X with respect to Euclidean norm, that is,

y = PX(x) = arg min_{z∈X}kx − zk.

The projection function PX(x) has the following property, called projection theorem [2], which is useful in our subsequent analysis.

Property 2.1 Let X be a nonempty closed convex subset of R^{n}. Then, for each z ∈ R^{n},
P_{X}(z) is the unique vector ¯z ∈ X such that (y − ¯z)^{T}(z − ¯z) ≤ 0, ∀y ∈ X.

Next, we recall some materials about first order differential equations (ODE):

˙

w(t) = H(w(t)), w(t_{0}) = w_{0} ∈ R^{n}, (6)
where H : R^{n}→ R^{n} is a mapping. We also introduce three kinds of stability that will be
discussed later. These materials can be found in usual ODE textbooks, e.g. [32].

Definition 2.3 A point w^{∗} = w(t^{∗}) is called an equilibrium point or a steady state of
the dynamic system (6) if H(w^{∗}) = 0. If there is a neighborhood Ω^{∗} ⊆ R^{n} of w^{∗} such
that H(w^{∗}) = 0 and H(w) 6= 0 ∀w ∈ Ω^{∗}\ {w^{∗}}, then w^{∗} is called an isolated equilibrium
point.

Lemma 2.1 Assume that H : R^{n} → R^{n} is a continuous mapping. Then, for any t_{0} > 0
and w0 ∈ R^{n}, there exists a local solution w(t) for (6) with t ∈ [t^{0}, τ ) for some τ > t0.
If, in addition, H is locally Lipschitz continuous at w0, then the solution is unique; if H
is Lipschitz continuous in R^{n}, then τ can be extended to ∞.

If a local solution defined on [t0, τ ) cannot be extended to a local solution on a larger
interval [t0, τ1), τ1 > τ, then it is called a maximal solution, and the interval [t0, τ ) is the
maximal interval of existence. Clearly, any local solution has an extension to a maximal
one. We denote [t0, τ (w0)) by the maximal interval of existence associated with w0.
Lemma 2.2 Assume that H : R^{n} → R^{n} is continuous. If w(t) with t ∈ [t^{0}, τ (w0)) is a
maximal solution and τ (w0) < ∞, then lim

t↑τ (w0)kw(t)k = ∞.

Definition 2.4 (Lyapunov Stability) Let w(t) be a solution for (6). An isolated equilib-
rium point w^{∗} is Lyapunov stable if for any w0 = w(t0) and any ε > 0, there exists a
δ > 0 such that if kw(t^{0}) − w^{∗}k < δ, then kw(t) − w^{∗}k < ε for all t ≥ t^{0}.

Definition 2.5 (Asymptotic Stability) An isolated equilibrium point w^{∗} is said to be
asymptotic stable if in addition to being Lyapunov stable, it has the property that if
kw(t^{0}) − w^{∗}k < δ, then w(t) → w^{∗} as t → ∞.

Definition 2.6 (Lyapunov function) Let Ω ⊆ R^{n} be an open neighborhood of ¯w. A
continuously differentiable function V : R^{n}→ R is said to be a Lyapunov function at the
state ¯w over the set Ω for equation (6) if

V ( ¯w) = 0, V (w) > 0, ∀w ∈ Ω \ { ¯w},

V (w) ≤ 0, ∀w ∈ Ω \ { ¯˙ w}. (7)

Lemma 2.3 (a) An isolated equilibrium point w^{∗} is Lyapunov stable if there exists a
Lyapunov function over some neighborhood Ω^{∗} of w^{∗}.

(b) An isolated equilibrium point w^{∗} is asymptotically stable if there exists a Lyapunov
function over some neighborhood Ω^{∗} of w^{∗} such that ˙V (w) < 0, ∀w ∈ Ω^{∗}\ {w^{∗}}.

Definition 2.7 (Exponential Stability) An isolated equilibrium point w^{∗} is exponentially
stable if there exists a δ > 0 such that arbitrary point w(t) of (6) with the initial condition
w(t0) = w0 and kw(t^{0}) − w^{∗}k < δ is well defined on [t^{0}, +∞) and satisfies

kw(t) − w^{∗}k ≤ ce^{−ωt}kw(t0) − w^{∗}k ∀t ≥ t0,
where c > 0 and ω > 0 are constants independent of the initial point.

### 3 Neural network model based on smoothed Fischer- Burmeister function

The smoothed Fischer-Burmeister function over the second-order cone defined below is used to construct a merit function by which the KKT system of SOCCVI is reformulated as an unconstrained smooth minimization problem. Furthermore, based on the mini- mization problem, we propose a neural network and study its stability in this section.

For any a = (a_{1}; a_{2}), b = (b_{1}; b_{2}) ∈ R × R^{n−1}, we define their Jordan product as
a · b = (a^{T}b; b1a2+ a1b2).

We denote a^{2} = a · a and |a| =√

a^{2}, where for any b ∈ K^{n}, √

b is the unique vector in
K^{n} such that b =√

b ·√ b.

Definition 3.1 A function φ : R^{n}×R^{n}→ R^{n}is called an SOC-complementarity function
if it satisfies

φ(a, b) = 0 ⇐⇒ a · b = 0, a ∈ K^{n}, b ∈ K^{n}.

A popular SOC-complementarity function is the Fischer-Burmeister function, which is semismooth [34] and defined as

φ_{FB}(a, b) = a^{2} + b^{2}1/2

− (a + b).

Then the smoothed Fischer-Burmeister function is given by
φ^{ε}_{FB}(a, b) = a^{2}+ b^{2} + ε^{2}e1/2

− (a + b) (8)

with ε ∈ R^{+} and e = (1, 0, · · · , 0)^{T} ∈ R^{n}.

The following lemma gives the gradient of φ^{ε}_{FB}. Since the proofs can be found in
[14, 34, 38], we here omit them.

Lemma 3.1 Let φ^{ε}_{FB} be defined as in (8) and ε 6= 0. Then, φ^{ε}FB is continuously differen-
tiable everywhere and

∇^{ε}φ^{ε}_{FB}(a, b) = e^{T}L^{−1}_{z} Lεe, ∇^{a}φ^{ε}_{FB}(a, b) = L^{−1}_{z} La− I, ∇^{b}φ^{ε}_{FB}(a, b) = L^{−1}_{z} Lb− I,
where z = (a^{2}+ b^{2}+ ε^{2}e)^{1/2}, I is identity mapping and La = a_{1} a^{T}_{2}

a2 a1In−1

for a =
(a1; a2) ∈ R × R^{n−1}.

Using Definition 3.1 and KKT condition described in [38], we can see that the KKT system (5) is equivalent to the following unconstrained smooth minimization problem:

min Ψ(w) := 1

2kS(w)k^{2}. (9)

Here Ψ(w), w = (ε, x, µ, λ) ∈ R^{1+n+l+m}, is a merit function, and S(w) is defined by

S(w) =

ε L(x, µ, λ)

−h(x)
φ^{ε}_{FB}(−g^{m}1(x), λm1)

...

φ^{ε}_{FB}(−g^{m}p(x), λmp)

,

with gmi(x), λmi ∈ R^{m}^{i}. In other words, Ψ(w) given in (9) is a smooth merit function for
the KKT system (5).

Based on the above smooth minimization problem (9), it is natural to propose the first neural network for solving the KKT system (5) of SOCCVI problem:

dw(t)

dt = −ρ∇Ψ(w(t)), w(t^{0}) = w0, (10)

where ρ > 0 is a scaling factor.

Remark 3.1 In fact, we can also adopt another merit function which is based on the FB function without the element ε. That is, we can define

S(x, µ, λ) =

L(x, µ, λ)

−h(x)
φ_{FB}(−g^{m}1(x), λm1)

...

φ_{FB}(−gmp(x), λ_{m}_{p})

. (11)

Then, the neural network model (10) could be obtained as well because kφFBk^{2} is smooth
[7]. However, it is observed that the gradient mapping ∇Ψ has more complicated formulas

because (−gmi(x))^{2} + λ^{2}_{m}_{i} may lie on the boundary of SOC, or interior of SOC, see [7,
34], which will cost more expensive numerical computations. Thus, the one dimensional
parameter ε in use not only has no influence on the main result, but also will simplify
the computational work.

To discuss properties of the neural network model (10), we make the following as- sumption which is used to avoid the singularity of ∇S(w), see [38].

Assumption 3.1 (a) the gradients {∇h^{j}(x)|j = 1, · · · , l} ∪ {∇g^{i}(x)|i = 1, · · · , m} are
linear independent.

(b) ∇^{x}L(x, µ, λ) is positive definite on the null space of the gradients {∇h^{j}(x)|j =
1, · · · , l}.

When SOCCVI problem corresponds to the KKT conditions of a convex second-
order cone program (CSOCP) problem as (4) where both h and g are linear, the above
Assumption 3.1(b) is indeed equivalent to the well-used condition ∇^{2}f (x) is positive
definite, e.g. [43, Corollary 1 ].

Proposition 3.1 Let Ψ : R^{1+n+l+m} → R^{+} be defined as in (9). Then, Ψ(w) ≥ 0 for
w = (ε, x, µ, λ) ∈ R^{1+n+l+m} and Ψ(w) = 0 if and only if (x, µ, λ) solves the KKT system
(5).

Proof. The proof is straightforward. 2

Proposition 3.2 Let Ψ : R^{1+n+l+m} → R^{+} be defined as in (9). Then, the following
results hold.

(a) The function Ψ is continuously differentiable everywhere with

∇Ψ(w) = ∇S(w)S(w), where

∇S(w) =

1 0 0 diag{∇εφ^{ε}_{FB}(−gmi(x), λ_{m}_{i})}^{p}i=1

0 ∇^{x}L(x, µ, λ)^{T} −∇h(x) −∇g(x)diag{∇^{g}_{mi}φ^{ε}_{FB}(−g^{m}i(x), λmi)}^{p}i=1

0 ∇h(x)^{T} 0 0

0 ∇g(x)^{T} 0 diag{∇^{λ}miφ^{ε}_{FB}(−g^{m}i(x), λmi)}^{p}i=1

.

(b) Suppose that assumption 3.1 holds. Then, ∇S(w) is nonsingular for any w ∈
R^{1+n+l+m}. Moreover, if (0, x, µ, λ) ∈ R^{1+n+l+m} is a stationary point of Ψ, then
(x, µ, λ) ∈ R^{n+l+m} is a KKT triple of the SOCCVI problem.

(c) Ψ(w(t)) is nonincreasing with respect to t.

Proof. Part(a) follows from the chain rule. For part(b), we know that ∇S(w) is non- singular if and only if the following matrix

∇^{x}L(x, µ, λ)^{T} −∇h(x) −∇g(x)diag{∇^{g}miφ^{ε}_{FB}(−g^{m}i(x), λmi)}^{p}i=1

∇h(x)^{T} 0 0

∇g(x)^{T} 0 diag{∇^{λ}miφ^{ε}_{FB}(−g^{m}i(x), λmi)}^{p}i=1

is nonsingular. In fact, from [38, Theorem 3.1] and [38, Proposition 4.1], the above
matrix is nonsingular and (x, µ, λ) ∈ R^{n+l+m} is a KKT triple of the SOCCVI problem
if (0, x, µ, λ) ∈ R^{1+n+l+m} is a stationary point of Ψ. It remains to show part(c). By the
definition of Ψ(w) and (10), it is not difficult to compute

dΨ(w(t))

dt = ∇Ψ(w(t))^{T}dw(t)

dt = −ρk∇Ψ(w(t))k^{2} ≤ 0. (12)
Therefore, Ψ(w(t)) is a monotonically decreasing function with respect to t. 2

Now, we are ready to analyze the behavior of the solution trajectory of (10) and establish properties of three kinds of stability for an isolated equilibrium point.

Proposition 3.3 (a) If (x, µ, λ) ∈ R^{n+l+m} is a KKT triple of SOCCVI problem, then
(0, x, µ, λ) ∈ R^{1+n+l+m} is an equilibrium point of (10).

(b) If Assumption 3.1 holds and (0, x, µ, λ) ∈ R^{1+n+l+m} is an equilibrium point of (10),
then (x, µ, λ) ∈ R^{n+l+m} is a KKT triple of SOCCVI problem.

Proof. (a) From Proposition 3.1 and (x, µ, λ) ∈ R^{n+l+m} being a KKT triple of SOCCVI
problem, it is clear that S(0, x, µ, λ) = 0. Hence, ∇Ψ(0, x, µ, λ) = 0. Besides, by
Proposition 3.2, we know that if ε 6= 0, then ∇Ψ(ε, x, µ, λ) 6= 0. This shows that
(0, x, µ, λ) is an equilibrium point of (10).

(b) It follows from (0, x, µ, λ) ∈ R^{1+n+l+m} being an equilibrium point of (10) that

∇Ψ(0, x, µ, λ) = 0. In other words, (0, x, µ, λ) is the stationary point of Ψ. Then, the result is a direct consequence of Proposition 3.2(b). 2

Proposition 3.4 (a) For any initial state w0 = w(t0), there exists exactly one maximal
solution w(t) with t ∈ [t0, τ (w_{0})) for the neural network (10).

(b) If the level set L(w^{0}) = {w ∈ R^{1+n+l+m}|Ψ(w) ≤ Ψ(w^{0})} is bounded, then τ(w^{0}) =
+∞.

Proof. (a) Since S is continuous differentiable, ∇S is continuous, and therefore, ∇S is bounded on a local compact neighborhood of w. That means ∇Ψ(w) is locally Lipschitz continuous. Thus, applying Lemma 2.1 leads to the desired result.

(b) This proof is similar to the proof of Case(i) in [5, Proposition 4.2]. 2

Remark 3.2 We wish to obtain the result that the level sets
L(Ψ, γ) := {w ∈ R^{1+n+l+m} | Ψ(w) ≤ γ}

are bounded for all γ ∈ R. However, we are not able to complete the argument. We suspect that there needs more subtle properties of F , h and g to finish it.

Next, we investigate the convergence of the solution trajectory of neural network (10)

Theorem 3.1 (a) Let w(t) with t ∈ [t^{0}, τ (w0)) be the unique maximal solution to (10).

If τ (w0) = +∞ and {w(t)} is bounded, then lim^{t→∞}∇Ψ(w(t)) = 0.

(b) If Assumption 3.1 holds and (ε, x, µ, λ) ∈ R^{1+n+l+m} is the accumulation point of the
trajectory w(t), then (x, µ, λ) ∈ R^{n+l+m} is a KKT triple of SOCCVI problem.

Proof. With Proposition 3.2(b) and (c) and Proposition 3.4, the arguments are exactly the same as those for [29, Corollary 4.3]. Thus, we omit them. 2

Theorem 3.2 Let w^{∗} be an isolated equilibrium point of the neural network (10). Then
the following results hold.

(a) w^{∗} is asymptotically stable.

(b) If Assumption 3.1 holds, then it is exponentially stable.

Proof. Since w^{∗} is an isolated equilibrium point of (10), there exists a neighborhood
Ω^{∗} ⊆ R^{1+n+l+m} of w^{∗} such that

∇Ψ(w^{∗}) = 0 and ∇Ψ(w) 6= 0 ∀w ∈ Ω^{∗}\ {w^{∗}}.

Next, we argue that Ψ(w) is indeed a Lyapunov function at x^{∗} over the set Ω^{∗} for
(10) by showing that the conditions in (7) are satisfied. First, notice that Ψ(w) ≥ 0.

Suppose that there is an ¯w ∈ Ω^{∗}\ {w^{∗}} such that Ψ( ¯w) = 0. Then, we can easily obtain
that ∇Ψ( ¯w) = 0, i.e., ¯w is also an equilibrium point of (10), which clearly contradicts
the assumption that w^{∗} is an isolated equilibrium point in Ω^{∗}. Thus, we prove that

Ψ(w) > 0 for any w ∈ Ω^{∗}\ {w^{∗}}. This together with (12) shows that the condition in
(7) are satisfied. Because w^{∗} is isolated, from (12), we have

dΨ(w(t))

dt < 0, ∀w(t) ∈ Ω^{∗}\ {w^{∗}}.

This implies that w^{∗} is asymptotically stable. Furthermore, if Assumption 3.1 holds, we
can obtain that ∇S is nonsingular. In addition, we have

S(w) = S(w^{∗}) + ∇S(w^{∗})(w − w^{∗}) + o(kw − w^{∗}k), ∀w ∈ Ω^{∗}\ {w^{∗}}. (13)
From kS(w(t))k being a monotonically decreasing function with respect to t and (13),
we can deduce that

kw(t) − w^{∗}k ≤ k(∇S(w^{∗}))^{−1}kkS(w(t)) − S(w^{∗})k + o(kw(t) − w^{∗}k)

≤ k(∇S(w^{∗}))^{−1}kkS(w(t0)) − S(w^{∗})k + o(kw(t) − w^{∗}k)

≤ k(∇S(w^{∗}))^{−1}k [k(∇S(w^{∗}))kkw(t^{0})−w^{∗}k+o(kw(t^{0})−w^{∗}k)]+o(kw(t) − w^{∗}k).

That is,

kw(t) − w^{∗}k−o(kw(t) − w^{∗}k) ≤ k(∇S(w^{∗}))^{−1}k [k(∇S(w^{∗}))kkw(t^{0})−w^{∗}k+o(kw(t^{0})−w^{∗}k)] .
The above inequality implies that the neural network (10) is also exponentially stable.

2

### 4 Neural network model based on projection func- tion

In this section, we present that the KKT triple of SOCCVI problem is equivalent to the solution of a projection formulation. Based on this, we introduce another neural network model for solving the projection formulation and analyze the stability conditions and convergence.

Define the function U : R^{n+l+m} → R^{n+l+m} and vector w in the following form:

U(w) =

L(x, µ, λ)

−h(x)

−g(x)

, w =

x µ λ

, (14)

where L(x, µ, λ) = F (x)+∇h(x)µ+∇g(x)λ is the Lagrange function. To avoid confusion,
we emphasize that, for any w ∈ R^{n+l+m}, we have

wi∈ R, if 1 ≤ i ≤ n + l,

wi∈ R^{m}^{i−(n+l)}, if n + l + 1 ≤ i ≤ n + l + p.

Then, we may write (14) as

U_{i} = (U(w))_{i} = (L(x, µ, λ))_{i}, w_{i} = x_{i}, i = 1, . . . , n,
Un+j = (U(w))n+j = −h^{j}(x), wn+j = µj, , j = 1, . . . , l,

Un+l+k = (U(w))n+l+k = −g^{k}(x) ∈ R^{m}^{k}, wn+l+k = λk∈ R^{m}^{k}, k = 1, . . . , p,

p

X

k=1

mk = m.

With this, the KKT conditions (5) can be recast as Ui = 0, i = 1, 2, . . . , n, n + 1, . . . , n + l,

hU^{J}, wJi = 0, UJ = (Un+l+1, Un+l+2, · · · , U^{n+l+p})^{T} ∈ K,
wJ = (wn+l+1, wn+l+2, · · · , w^{n+l+p})^{T} ∈ K.

(15)

Thus, (x^{∗}, µ^{∗}, λ^{∗}) is a KKT triple for (1) if and only if (x^{∗}, µ^{∗}, λ^{∗}) is a solution to (15).

It is well known that the nonlinear complementarity problem, which is denoted by
NCP(F, K) and to find an x ∈ R^{n} such that

x ∈ K, F (x) ∈ K and hF (x), xi = 0

where K is a closed convex set of R^{n}, is equivalent to the following VI(F, K) problem:

finding an x ∈ K such that

hF (x), y − xi ≥ 0 ∀y ∈ K.

Furthermore, if K = R^{n}, then NCP(F, K) becomes the system of nonlinear equations
F (x) = 0.

Based on the above, solution of (15) is equivalent to solution of the following VI problem: find w ∈ K such that

hU(w), v − wi ≥ 0, ∀v ∈ K, (16)

where K = R^{n+l}× K. In addition, by applying the Property 2.1, its solution is equivalent
to solution of below projection formulation

P_{K}(w − U(w)) = w with K = R^{n+l} × K, (17)
where function U and vector w are defined in (14). Now, according to (17), we give the
following neural network:

dw

dt = ρ{PK(w − U(w)) − w}, (18)

where ρ > 0. Note that K is a closed and convex set. For any w ∈ R^{n+l+m}, P_{K} means
P_{K}(w) = [P_{K}(w_{1}), P_{K}(w_{2}), · · · , PK(w_{n+l}), P_{K}(w_{n+l+1}), P_{K}(w_{n+l+2}), · · · , PK(w_{n+l+p})],

where

P_{K}(w_{i}) = w_{i}, i = 1, · · · , n + l,

P_{K}(wn+l+j) = [λ1(wn+l+j)]+· u^{(1)}^{w}n+l+j+ [λ2(wn+l+j)]+· u^{(2)}^{w}n+l+j, j = 1, · · · , p.

Here, for the sake of simplicity, we denote the vector wn+l+j by v for the moment, and
[·]+ is the scalar projection, λ_{1}(v), λ_{2}(v) and u^{(1)}v , u^{(2)}v are the spectral values and the
associated spectral vectors of v = (v1; v2) ∈ R × R^{m}^{j}^{−1}, respectively, given by

( λi(v) = v1+ (−1)^{i}kv^{2}k,
u^{(i)}v = ^{1}_{2}

1, (−1)^{i v}_{kv}^{2}_{2}_{k}
,
for i = 1, 2, see [7, 34].

The dynamic system described by (18) can be recognized as a recurrent neural network with a single-layer structure. To analyze the stability conditions of (18), we need the following lemmas and proposition.

Lemma 4.1 If the gradient of L(x, µ, λ) is positive semi-definite (respectively, positive definite), then the gradient of U in (14) is positive semi-definite (respectively, positive definite).

Proof. Since we have

∇U(x, µ, λ) =

∇^{x}L^{T}(x, µ, λ) −∇h(x) −∇g(x)

∇^{T}h(x) 0 0

∇^{T}g(x) 0 0

,

for any nonzero vector d = (p^{T}, q^{T}, r^{T})^{T} ∈ R^{n+l+m}, we can obtain that

d^{T}∇U(x, µ, λ)d = p^{T} q^{T} r^{T}

∇^{x}L^{T}(x, µ, λ) −∇h(x) −∇g(x)

∇^{T}h(x) 0 0

∇^{T}g(x) 0 0

p q r

= p^{T}∇^{x}L(x, µ, λ)p.

This leads to the desired results. 2

Proposition 4.1 For any initial point w0 = (x0, µ0, λ0) with λ0 := λ(t0) ∈ K, there exist a unique solution w(t) = (x(t), µ(t), λ(t)) for neural network (18), Moreover, λ(t) ∈ K.

Proof. For simplicity, we assume K = K^{m}. The analysis can be carried over to the
general case where K is the Cartesian product of second-order cones. Since F, h, g are
continuous differentiable, the function

F (w) := P_{K}(w − U(w)) − w with K = R^{n+l}× K^{m} (19)
is semi-smooth and Lipschitz continuous. Thus, there exists a unique solution w(t) =
(x(t), µ(t), λ(t)) for neural network (18). Therefore, it remains to show that λ(t) ∈ K^{m}.
For convenience, we denote λ(t) := (λ1(t), λ2(t)) ∈ R × R^{m−1}. To complete the proof,
we need to verify two things: (i) λ1(t) ≥ 0 and (ii) kλ^{2}(t)k ≤ λ^{1}(t). First, from (18), we

have dλ

dt + ρλ(t) = ρP_{K}^{m}(λ + g(x)).

The solution of the above first-order ordinary differential equation is
λ(t) = e^{−ρ(t−t}^{0}^{)}λ(t0) + ρe^{−ρt}

Z t t0

ρe^{ρs}PK^{m}(λ + g(x))ds. (20)

If we let λ(t0) := (λ1(t0), λ2(t0)) ∈ R × R^{m−1} and denote PK^{m}(λ + g(x)) as z(t0) :=

(z1(t0), z2(t0)), then (20) leads to

λ1(t) = e^{−ρ(t−t}^{0}^{)}λ1(t0) + ρe^{−ρt}
Z t

t0

ρe^{ρs}z1(s)ds, (21)

λ2(t) = e^{−ρ(t−t}^{0}^{)}λ2(t0) + ρe^{−ρt}
Z t

t0

ρe^{ρs}z2(s)ds. (22)
Due to both λ(t0) and z(t) belong to K^{m}, there have λ1(t0) ≥ 0, kλ^{2}(t0)k ≤ λ^{1}(t0) and
kz^{2}(t)k ≤ z^{1}(t). Therefore, λ1(t) ≥ 0 since both terms in the right-hand side of (21) are
nonnegative. In addition,

kλ^{2}(t)k ≤ e^{−ρ(t−t}^{0}^{)}kλ^{2}(t0)k + ρe^{−ρt}Rt

t0ρe^{ρs}kz^{2}(s)kds

≤ e^{−ρ(t−t}^{0}^{)}λ1(t0) + ρe^{−ρt}Rt

t0ρe^{ρs}z1(s)ds

= λ1(t),
which implies that λ(t) ∈ K^{m} 2

Lemma 4.2 Let U(w), F (w) be defined as in (14) and (19), respectively. Suppose w^{∗} =
(x^{∗}, µ^{∗}, λ^{∗}) is an equilibrium point of neural network (18) with (x^{∗}, µ^{∗}, λ^{∗}) being an KKT
triple of SOCCVI problem. Then, the following inequality holds:

(F (w) + w − w^{∗})^{T}(−F (w) − U(w)) ≥ 0. (23)

Proof. Notice that

(F (w) + w − w^{∗})^{T}(−F (w) − U(w))

= [−w + PK(w − U(w)) + w − w^{∗}]^{T}[w − PK(w − U(w)) − U(w)]

= [−w^{∗}+ P_{K}(w − U(w))]^{T}[w − PK(w − U(w)) − U(w)]

= −[w^{∗}− PK(w − U(w))]^{T}[w − U(w) − PK(w − U(w))].

Since w^{∗} ∈ K, applying Property 2.1 gives

[w^{∗}− PK(w − U(w))]^{T}[w − U(w) − PK(w − U(w))] ≤ 0.

Thus, we have

(F (w) + w − w^{∗})^{T}(−F (w) − U(w)) ≥ 0.

This completes the proof. 2

We now show the stability and convergence issues regarding neural network (18).

Theorem 4.1 If ∇^{x}L(w) is positive semi-definite (respectively, positive definite), the
the solution of neural network (18) with initial point w0 = (x0, µ0, λ0) where λ0 ∈ K is
Lyapunov stable (respectively, asymptotically stable). Moreover, the solution trajectory
of neural network (18) is extendable to the global existence.

Proof. Again, for simplicity, we assume K = K^{m}. From Proposition 4.1, there exists
a unique solution w(t) = (x(t), µ(t), λ(t)) for neural network (18) and λ(t) ∈ K^{m}. Let
w^{∗} = (x^{∗}, µ^{∗}, λ^{∗}) be an equilibrium point of neural network (18). We define a Lyapunov
function as below:

V (w) := V (x, µ, λ) := −U(w)^{T}F (w) − 1

2kF (w)k^{2}+ 1

2kw − w^{∗}k^{2}. (24)
From [12, Theorem 3.2], we know that V is continuously differentiable with

∇V (w) = U(w) − [∇U(w) − I]F (w) + (w − w^{∗}).

It is also trivial that V (w^{∗}) = 0. Then, we have
dV (w(t))

dt = ∇V (w(t))^{T}dw

= {U(w) − [∇U(w) − I]F (w) + (w − wdt ^{∗})}^{T}ρF (w)

= ρ{[U(w) + (w − w^{∗})]^{T}F (w) + kF (w)k^{2}− F (w)^{T}∇U(w)F (w)}.

Inequality (23) in Lemma 4.2 implies

[U(w) + (w − w^{∗})]^{T}F (w) ≤ −U(w)^{T}(w − w^{∗}) − kF (w)k^{2},

which yields dV (w(t))

≤ ρ{−U(w)dt ^{T}(w − w^{∗}) − F (w)^{T}∇U(w)F (w)}

= ρ{−U(w^{∗})^{T}(w − w^{∗}) − (U(w) − U(w^{∗}))^{T}(w − w^{∗}) − F (w)^{T}∇U(w)F (w)}.

(25)

Note that w^{∗} is the solution of the variational inequality (16). Since w ∈ K, we therefore
obtain −U(w^{∗})^{T}(w − w^{∗}) ≤ 0. Because U(w) is continuous differentiable and ∇U(w) is
positive semi-definite, by [33, Theorem 5.4.3], we obtain that U(w) is monotone. Hence,
we have −(U(w) − U(w^{∗}))^{T}(w − w^{∗}) ≤ 0 and −F (w)^{T}∇U(w)F (w) ≤ 0. The above
discussions lead to ^{dV (w(t))}_{dt} ≤ 0. Also, by [33, Theorem 5.4.3], we know that if ∇U(w) is
positive definite, then U(w) is strictly monotone, which implies ^{dV (w(t))}_{dt} < 0 in this case.

In order to obtain V (w) is a Lyapunov function and w^{∗} is Lyapunov stable, we will
show the following inequality:

−U(w)^{T}F (w) ≥ kF (w)k^{2}. (26)

To see this, we first observe that

kF (w)k^{2}+ U(w)^{T}F (w) = [w − PK(w − U(w))]^{T}[w − U(w) − PK(w − U(w))].

Since w ∈ K, applying Property 2.1 again, there holds

[w − PK(w − U(w))]^{T}[w − U(w) − PK(w − U(w))] ≤ 0,

which yields the desired inequality (26). By combining equation (24) and (26), we have V (w) ≥ 1

2kF (w)k^{2}+ 1

2kw − w^{∗}k^{2},

which says V (w) > 0 if w 6= w^{∗}. Hence V (w) is indeed a Lyapunov function and w^{∗} is
Lyapunov stable. Furthermore, if ∇xL(w) is positive definite, we have w^{∗} is asymptoti-
cally stable. Moreover, it holds that

V (w_{0}) ≥ V (w) ≥ 1

2kw − w^{∗}k^{2} for t ≥ t0, (27)
which tells us the solution trajectory w(t) is bounded. Hence, it can be extended to
global existence. 2

Theorem 4.2 Let w^{∗} = (x^{∗}, µ^{∗}, λ^{∗}) be an equilibrium point of (18). If ∇^{x}L(w) is positive
definite, the solution of neural network (18) with initial point w0 = (x0, µ0, λ0) where
λ_{0} ∈ K is globally convergent to w^{∗} and has finite convergence time.

Proof. From (27), the level set

L(w0) := {w|V (w) ≤ V (w0)}

is bounded. Then, the Invariant Set Theorem [15] implies the solution trajectory w(t) converges to θ as t → +∞ where θ is the largest invariant set in

Λ =

w ∈ L(w^{0})

dV (w(t)) dt = 0

.

We will show that dw/dt = 0 if and only if dV (w(t))/dt = 0 which yields that w(t)
converges globally to the equilibrium point w^{∗} = (x^{∗}, µ^{∗}, λ^{∗}). Suppose dw/dt = 0, then
it is clear that dV (w(t))/dt = ∇V (w)^{T}(dw/dt) = 0. Let ˆw = (ˆx, ˆµ, ˆλ) ∈ Λ which says
dV ( ˆw(t))/dt = 0. From (23), we know that

dV ( ˆw(t))/dt ≤ ρ{(−U( ˆw) − U(w^{∗}))^{T}( ˆw − w^{∗}) − F ( ˆw)^{T}∇U( ˆw)F ( ˆw)}.

Both terms inside the big parenthesis are nonpositive as shown in Theorem 4.1, so (U( ˆw)−

U(w^{∗}))^{T}( ˆw − w^{∗}) = 0, F ( ˆw)^{T}∇U( ˆw)F ( ˆw) = 0. The condition ∇xL(w) being positive
definite leads to ∇U( ˆw) being positive definite. Hence,

F ( ˆw) = − ˆw + P_{K}( ˆw − U( ˆw)) = 0,

which is equivalent to d ˆw/dt = 0. From the above, w(t) converges globally to the
equilibrium point w^{∗}= (x^{∗}, µ^{∗}, λ^{∗}). Moreover, with Theorem 4.1 and following the same
argument as in [43, Theorem 2], the neural network (18) has finite convergence time.

2

### 5 Simulations

To demonstrate effectiveness of the proposed neural networks, some illustrative SOCCVI problems are tested. The numerical implementation is coded by Matlab 7.0 and the ordinary differential equation solver adopted is ode23, which uses Runge-Kutta (2; 3) formula. In the following tests, the parameter ρ in both neural networks is set to be 1000.

Example 5.1 Consider the SOCCVI problem (1)-(2) where

F (x) =

2x1+ x2+ 1
x1+ 6x2− x^{3}− 2

−x2+ 3x_{3}−^{6}_{5}x_{4}+ 3

−^{6}_{5}x3+ 2x4+ ^{1}_{2}sin x4cos x5sin x6+ 6

1

2cos x4sin x5sin x6+ 2x5−^{5}_{2}

−^{1}2 cos x4cos x5cos x6+ 2x6 +^{1}_{4}cos x6sin x7cos x8+ 1

1

4sin x_{6}cos x_{7}cos x_{8}+ 4x_{7}− 2

−^{1}_{4}sin x6sin x7sin x8+ 2x8+^{1}_{2}

and

C = {x ∈ R^{8} : −g(x) = x ∈ K^{3}× K^{3}× K^{2}}.

This problem has an approximate solution

x^{∗} = (0.3820, 0.1148, −0.3644, 0.0000, 0.0000, 0.0000, 0.5000, −0.2500)^{T}.
It can be verified that the Lagrangian function for this example is

L(x, µ, λ) = F (x) − λ and the gradient of the Lagrangian function is

∇L(x, µ, λ) = ∇F (x) I8×8

,

where I is the identity mapping and ∇F (x) means the gradient of F (x). We use the
proposed neural networks with smoothed FB and projection functions, respectively, to
solve the problem whose trajectories are depicted in Figures 1 and 2. The simulation
results show that both trajectories are globally convergent to x^{∗} and the neural network
with projection function converges to x^{∗} quicker than that with smoothed FB function.

0 10 20 30 40 50 60 70 80

−1.5

−1

−0.5 0 0.5 1

Time (ms)

Trajectories of x(t)

x1 x2

x3

x4 x5

x6

x8 x7

Figure 1: Transient behavior of neural network with smoothed FB function in Example 5.1.

0 5 10 15 20 25

−3

−2.5

−2

−1.5

−1

−0.5 0 0.5 1 1.5

Time (ms)

Trajectories of x(t)

x1 x2

x3

x4 x5

x6

x8 x7

Figure 2: Transient behavior of the neural network with projection function in Example 5.1.

Example 5.2 [38, Example 5.1] We consider the following SOCCVI problem:

1

2Dx, y − x

≥ 0, ∀y ∈ C where

C = {x ∈ R^{n}: Ax − a = 0, Bx − b 0},

D is an n × n symmetric matrix, A and B are l × n and m × n matrices, respectively, d is an n × 1 vector, a and b are l × 1 and m × 1 vectors with l + m ≤ n, respectively.

In fact, we can determine the data a, b, A, B and D randomly. However, as in [38, Example 5.1], we set the data as follows:

D = [Dij]n×n, where Dij =

2, i = j 1, |i − j| = 1 0, otherwise

,

A = [Il×l 0l×(n−l)]l×n, B = [0m×(n−m) Im×m]m×n, a = 0l×1, b = (em1, em2, · · · , e^{m}^{p})^{T},
where emi = (1, 0, · · · , 0)^{T} ∈ R^{m}^{i} and l + m ≤ n. Clearly, A and B are full row rank and
rank([A^{T} B^{T}]) = l + m.

In the simulation, the parameters l, m, and n are set to be 3,3, and 6, respectively. The
problem has an solution x^{∗} = (0, 0, 0, 0, 0, 0)^{T}. It can be verified that the Lagrangian
function for this example is

L(x, µ, λ) = 1

2Dx + A^{T}µ + B^{T}λ.

Note that ∇^{x}L(x, µ, λ) is positive definite. We know from Theorems 3.1 and 4.2 that
both proposed neural networks are globally convergent to the KKT triple of the SOC-
CVI problem. Figures 3 and 4 depict the trajectories of Example 5.2 obtained using
the proposed neural networks. The simulation results show that both neural networks
are effective in the SOCCVI problem and the neural network with projection function
converges to x^{∗} quicker than that with smoothed FB function.

0 20 40 60 80 100 120 140 160

−0.2

−0.15

−0.1

−0.05 0 0.05 0.1 0.15 0.2 0.25

Time (ms)

Trajectories of x(t)

x5

x1 x3

x2

x4 x6

Figure 3: Transient behavior of neural network with smoothed FB function in Example 5.2.

Example 5.3 Consider the SOCCVI problem (1)-(2) where

F (x) =

x3exp(x1x3) + 6(x1+ x2)
6(x1+ x2) + 2(2x2− x^{3})

p1 + (2x2− x^{3})^{2}
x1exp(x1x3) − 2x_{2}− x3

p1 + (2x2− x3)^{2}
x4

x5

0 5 10 15 20 25 30

−0.4

−0.3

−0.2

−0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

Time (ms)

Trajectories of x(t)

x6 x4 x1 x3 x5

x2

Figure 4: Transient behavior of neural network with projection function in Example 5.2.

and

C = {x ∈ R^{5} : h(x) = 0, −g(x) ∈ K^{3}× K^{2}},
with

h(x) = −62x^{3}1+ 58x2+ 167x^{3}_{3}− 29x^{3}− x^{4}− 3x^{5}+ 11,

g(x) =

−3x^{3}1− 2x^{2}+ x3− 5x^{3}3

5x^{3}_{1}− 4x2+ 2x_{3}− 10x^{3}3

−x^{3}

−x^{4}

−3x^{5}

.

This problem has an approximate solution x^{∗} = (0.6287, 0.0039, −0.2717, 0.1761, 0.0587)^{T}.
Example 5.4 Consider the SOCCVI problem (1)-(2) where

F (x) =

4x1 − sin x^{1}cos x2+ 1

− cos x^{1}sin x2+ 6x2+^{9}_{5}x3+ 2

9

5x2+ 8x3+ 3 2x4+ 1

and

C =

x ∈ R^{4} : h(x) =x^{2}_{1}−10^{1}x2x3+ x3

x^{2}_{3}+ x4

= 0, −g(x) =x_{1}
x2

∈ K^{2}

.

This problem has an approximate solution x^{∗} = (0.2391, −0.2391, −0.0558, −0.0031)^{T}.
The neural network (10) based on smoothed FB function can solve Examples 5.3-5.4
sucessfully, see Figures 5-6, whereas the neural network (18) based on projection function
fails to solve them. This is because that ∇^{x}L(x, µ, λ) is not always positive definite in
Examples 5.3 and 5.4. Hence, the neural network with projection function is not effective
in these two problems. To the contrast, though there is no guarantee that the Assumption
3.1 (b) holds, the neural network with smoothed FB function is asymptotically stable from
Theorem 3.2. Figures 5 and 6 depict the trajectories obtained using the neural network
with the smoothed FB function for Examples 5.3 and 5.4, respectively. The simulation
results show that each trajectory converges to the desired isolated equilibrium point
which is exactly the approximate solution of Example 5.3 and Example 5.4, respectively.

0 10 20 30 40 50 60

−0.4

−0.2 0 0.2 0.4 0.6 0.8 1

Time (ms)

Trajectories of x(t)

x1

x2

x3 x4 x5

Figure 5: Transient behavior of neural network with smoothed FB function in Example 5.3.

Example 5.5 Consider the nonlinear convex SOCP [24] given by
min exp(x1− x^{3}) + 3(2x1− x^{2})^{4}+p1 + (3x2 + 5x3)^{2}

s.t. − g(x) =

4x1+ 6x2+ 3x3− 1

−x1+ 7x_{2}− 5x3 + 2
x1

x2

x3

∈ K^{2} × K^{3}

0 50 100 150

−0.5

−0.4

−0.3

−0.2

−0.1 0 0.1 0.2 0.3 0.4 0.5

Time (ms)

Trajectories of x(t)

x1

x2 x3 x4

Figure 6: Transient behavior of neural network with smoothed FB function in Example 5.4.

The approximate solution of this problem is x^{∗} = (0.2324, −0.07309, 0.2206)^{T}. As men-
tioned in Section 1, the CSOCP in Example 5.5 can be transformed into an equivalent
SOCCVI problem. There are neural network models proposed for CSOCP in [28]. We
here try a different approach for it. In other words, we use the proposed neural net-
works with the smoothed FB and projection functions, respectively, to solve the problem
whose trajectories are depicted in Figures 7 and 8. From the simulation results, we see
that the neural network with smoothed FB function converges very slowly and it is not
clear whether it converges to the solution in finite time or not. In view of these, to
solve CSOCP, it seems better to apply the models introduced in [28] directly instead of
transforming CSOCP into SOCCVI problem.

The simulation results of Examples 5.1, 5.2 and 5.5 tell us that the neural network with
projection function converges to x^{∗} quicker than that with the smoothed FB function.

In general, the neural network with projection function has lower model complexity than
that with the smoothed FB function. Hence, the neural network with projection function
is preferable to the neural network with the smoothed FB function when both can globally
converge to the solution of SOCCVI problem. On the other hand, from Examples 5.3
and 5.4, the neural network with smoothed FB function seems better for use when the
positive semidefinite condition of ∇^{x}L(x, µ, λ) is not satisfied.

0 50 100 150 200

−0.3

−0.2

−0.1 0 0.1 0.2 0.3 0.4 0.5

Time (ms)

Trajectories of x(t)

x1

x2 x3

Figure 7: Transient behavior of neural network with smoothed FB function in Example 5.5.

0 20 40 60 80 100 120

−0.2

−0.1 0 0.1 0.2 0.3 0.4 0.5

Time (ms)

Trajectories of x(t)

x1

x2 x3

Figure 8: Transient behavior of neural network with projection function in Example 5.5.

### 6 Conclusions

In this paper, We use the proposed neural networks with smoothed Fischer-Burmeister
and projection functions to solve the SOCCVI problems. The first neural network uses
the Fischer-Burmeister (FB) function to achieve an unconstrained minimization which
is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit func-
tion is a Lyapunov function and this neural network is asymptotically stable. Under
Assumption 3.1, we prove that if (ε, x, µ, λ) ∈ R^{1+n+l+m} is the accumulation point of the
trajectory, then (x, µ, λ) ∈ R^{n+l+m} is a KKT triple of SOCCVI problem and the neural
network is exponentially stable. The second neural network is introduced for solving a
projection formulation whose solutions coincide with the KKT triples of SOCCVI prob-
lem under the positive semidefinite condition of ∇^{x}L(x, µ, λ). Its Lyapunov stability and
global convergence are proved. Simulations show that both neural networks have merits
of their own.

Acknowledgements. The authors thank the referees for their carefully reading this paper and helpful suggestions.

### References

[1] M. Aghassi, D. Bertsimas, and G. Perakis, Solving asymmetric variational inequalities via convex optimization, Operations Research Letters, 34(2006), 481–490.

[2] D. P. Bertsekas, Nonlinear programming, Athena Scientific, 1995.

[3] J.-S. Chen, The semismooth-related properties of a merit function and a decent method for the nonlinear complementarity problem, Journal of Global Optimization, 36(2006), 565–580.

[4] J.-S. Chen, H.-T. Gao, and S.-H. Pan, An R-linearly convergent derivative-free algorithm for nonlinear complementarity problems based on the generalized Fischer- Burmeister merit function, Journal of Computational and Applied Mathematics, 232(2009), 455–471.

[5] J.-S. Chen, C.-H. Ko, and S.-H. Pan, A neural network based on the general- ized Fischer-Burmeister function for nonlinear complementarity problems, Informa- tion Sciences, vol. 180(2010), 697–711.

[6] J.-S. Chen and S.-H. Pan, A family of NCP functions and a descent method for the nonlinear complementarity problem, Computational Optimization and Applications, 40(2008), 389–404.

[7] J.-S. Chen and P. Tseng, An unconstrained smooth minimization reformula- tion of the second-order cone complementarity problem, Mathematical Programming, 104(2005), 293–327.

[8] X. Chen, L. Qi, and D. Sun, Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities, Mathematics of Computation, 67(222)(1998), 519–540.

[9] C. Dang, Y. Leung, X. Gao, and K. Chen, Neural networks for nonlinear and mixed complementarity problems and their applications, Neural Networks, 17(2004), 271–283.

[10] S. Effati, A. Ghomashi, and A. R. Nazemi, Applocation of projection neural network in solving convex programming problems, Applied Mathematics and Compu- tation, 188(2007), 1103–1114.

[11] S. Effati and A. R. Nazemi, Neural network and its application for solving linear and quadratic programming problems, Applied Mathematics and Computation, 172(2006), 305–331.

[12] M. Fukushima, Equivalent differentiable optimization problems and descent meth- ods for asymmetric variational inequality problems, Mathematical Programming, 53(1)(1992), 99–110.

[13] F. Facchinei, J. Pang, Finite-dimensional Variational Inequalities and Comple- mentarity Problems, Springer, New York, 2003.

[14] M. Fukushima, Z.-Q. Luo, and P. Tseng, Smoothing functions for second- order-cone complimentarity problems, SIAM Journal on Optimization, 12(2002), 436–

460.

[15] R. Golden, Mathematical methods for neural network analysis and design, The MIT Press, 1996.

[16] J. Hammond, Solving asymmetric variational inequality problems and systems of equations with generalized nonlinear programming algorithms, Ph.D. Dissertation, De- partment of Mathematics, MIT, Cambridge, 1984.

[17] Q. Han, L.-Z. Liao, H. Qi, and L. Qi, Stability analysis of gradient-based neural networks for optimiation problems, Journal of Global Optimization, 19(2001), 363–

381.

[18] P. Harker and J.-S. Pang, Finite-dimensional variational inequality and non- linear complementarity problems: a survey of theory, algorithms and applications, Mathematical Programming, 48(1990), 161–220.

[19] P. Hartman and G. Stampacchia, On some nonlinear elliptic differential func- tional equations, Acta Mathematica, 115(1966), 271–310.

[20] J. J. Hopfield and D. W. Tank, Neural computation of decision in optimization problems, Biological Cybernetics, 52(1985), 141–152.

[21] X. Hu and J. Wang, A recurrent neural network for solving nonlinear con- vex programs subject to linear constraints, IEEE Transactions on Neural net- work,16(3)(2005), 379–386.

[22] X. Hu and J. Wang, Solving pseudomonotone variational inequalities and pseudo- convex optimization problems using the projection neural network, IEEE Transactions on Neural Network, 17(2006), 1487–1499.

[23] X. Hu and J. Wang, A recurrent neural network for solving a class of general variational inequalities, IEEE Transactions on Systems, Man, and Cybernetics-B, 37(2007), 528–539.

[24] C. Kanzow, I. Ferenczi, and M. Fukushima, On the local convergence of semismooth Newton methods for linear and nonlinear second-order cone programs without strict complementarity, SIAM Journal on Optimization, 20(2009), 297–320.

[25] M. P. Kennedy and L. O. Chua, Neural network for nonlinear programming, IEEE Transactions on Circuits and Systems, 35(1988), 554–562.

[26] Y. Kanno, J. A. C. Martins, and A. Pinto da Costa, Three-dimensional quasi-static frictional contact by using second-order cone linear complementarity prob- lem, International Journal for Numerical Methods in Engneering, 65(2006), 62–83.

[27] D. Kinderlehrer and G. Stampacchia, An introduction to variational inequal- ities and their applications, Academic Press, New York, 1980.

[28] C.-H. Ko, J.-S. Chen, and C.-Y. Yang, A recurrent neural network for solving nonlinear second-order cone programs, submitted manuscript, 2009.

[29] L.-Z. Liao, H. Qi, and L. Qi, Solving nonlinear complementarity problems with neural networks: a reformulation method approach, Jorunal of Computational and Applied Mathematics, 131(2001), 342–359.

[30] J. Lions and G. Stampacchia, Variational inequalities, Communications on Pure and Applied Mathematics, 20(1967), 493–519.

[31] O. Mancino and G. Stampacchia, Convex programming and variational inequal- ities, Journal of Optimization Theory and Applications, 9(1972), 3–23.

[32] R. K. Miller and A. N. Michel, Ordinary differential equations, Academic Press, 1982.