DOI 10.1007/s10589-010-9359-x

**Neural networks for solving second-order cone** **constrained variational inequality problem**

**Juhe Sun· Jein-Shan Chen · Chun-Hsu Ko**

Received: 25 April 2010 / Published online: 6 October 2010

© Springer Science+Business Media, LLC 2010

**Abstract In this paper, we consider using the neural networks to efficiently solve**
the second-order cone constrained variational inequality (SOCCVI) problem. More
specifically, two kinds of neural networks are proposed to deal with the Karush-Kuhn-
Tucker (KKT) conditions of the SOCCVI problem. The first neural network uses the
Fischer-Burmeister (FB) function to achieve an unconstrained minimization which
is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit
function is a Lyapunov function and this neural network is asymptotically stable.

The second neural network is introduced for solving a projection formulation whose solutions coincide with the KKT triples of SOCCVI problem. Its Lyapunov stability and global convergence are proved under some conditions. Simulations are provided to show effectiveness of the proposed neural networks.

**Keywords Second-order cone**· Variational inequality · Fischer-Burmeister
function· Neural network · Lyapunov stable · Projection function

J. Sun is also affiliated with Department of Mathematics, National Taiwan Normal University.

J.-S. Chen is member of Mathematics Division, National Center for Theoretical Sciences, Taipei Office. The author’s work is partially supported by National Science Council of Taiwan.

J. Sun

School of Science, Shenyang Aerospace University, Shenyang 110136, China e-mail:juhesun@abel.math.ntnu.edu.tw

J.-S. Chen (

^{)}

Department of Mathematics, National Taiwan Normal University, Taipei 11677, Taiwan e-mail:jschen@math.ntnu.edu.tw

C.-H. Ko

Department of Electrical Engineering, I-Shou University, Kaohsiung 840, Taiwan e-mail:chko@isu.edu.tw

**1 Introduction**

Variational inequality (VI) problem, which was introduced by Stampacchia and his collaborators [19,29,30,35,36], has attracted much attention from researchers of engineering, mathematics, optimization, transportation science, and economics com- munities, see [1,25,26]. It is well known that VIs subsume many other mathematical problems, including the solution of systems of equations, complementarity problems, and a class of fixed point problems. For a complete discussion and history of the finite VI problem and its associated solution methods, we refer the interested readers to the excellent survey text by Facchinei and Pang [13], the monograph by Patriksson [34], the survey article by Harker and Pang [18], the Ph.D. thesis of Hammond [16] and the references therein.

In this paper, we are interested in solving the second-order cone constrained varia-
tional inequality (SOCCVI) problem whose constraints involve the Cartesian product
*of second-order cones (SOCs). The problem is to find x∈ C satisfying*

*F (x), y − x ≥ 0, ∀y ∈ C,* (1)

*where the set C is finitely representable as*

*C= {x ∈ R*^{n}*: h(x) = 0, −g(x) ∈ K}.* (2)
Here*·, · denotes the Euclidean inner product, F : R** ^{n}*→ R

^{n}*, h*: R

*→ R*

^{n}

^{l}*and g*: R

*→ R*

^{n}*are continuously differentiable functions and*

^{m}*K is a Cartesian product of*second-order cones (or Lorentz cones), expressed as

*K = K*^{m}^{1}*× K*^{m}^{2}*× · · · × K*^{m}^{p}*,* (3)
*where l≥ 0, m*1*, m*2*, . . . , m**p**≥ 1, m*1*+ m*2*+ · · · + m**p**= m, and*

*K*^{m}^{i}*:= {(x**i1**, x**i2**, . . . , x**im*_{i}*)** ^{T}* ∈ R

^{m}

^{i}*| (x*

*i2*

*, . . . , x*

*im*

_{i}*) ≤ x*

*i1*}

with*· denoting the Euclidean norm and K*^{1}the set of nonnegative realsR+*.*A spe-
cial case of (3) is*K = R*^{n}_{+}*,*namely the nonnegative orthant inR^{n}*,*which corresponds
*to p= n and m*1*= · · · = m**p**= 1. When h is affine, an important special case of*
the SOCCVI problem corresponds to the KKT conditions of the convex second-order
cone program (CSOCP):

min *f (x)*

*s.t.* *Ax= b, −g(x) ∈ K,* (4)

*where A*∈ R^{l}^{×n}*has full row rank, b*∈ R^{l}*, g*: R* ^{n}*→ R

^{m}*and f*: R

*→ R. Further-*

^{n}*more, when f is a convex twice continuously differentiable function, problem (4) is*

*equivalent to the following SOCCVI problem: Find x∈ C satisfying*

*∇f (x), y − x ≥ 0, ∀y ∈ C,*
where

*C= {x ∈ R*^{n}*: Ax − b = 0, −g(x) ∈ K}.*

For solving the constrained variational inequalities and complementary problems (CP), many computational methods have been proposed, see [3,4,6,8,13,39] and references therein. These methods include the method based on merit function, in- terior method, Newton method, nonlinear equation method, projection method and its variant versions. Another class of techniques for solving the VI problem exploits the fact that the KKT conditions of a VI problem comprise a mixed complementar- ity problem (MiCP), involving both equations and nonnegativity constraints. In other words, the SOCCVI problem can be solved by analyzing its KKT conditions:

⎧⎪

⎨

⎪⎩

*L(x, μ, λ)= 0,*

*g(x), λ = 0,* *−g(x) ∈ K, λ ∈ K,*
*h(x)= 0,*

(5)

*where L(x, μ, λ)= F (x) + ∇h(x)μ + ∇g(x)λ is the variational inequality La-*
*grangian function, μ*∈ R^{l}*and λ*∈ R^{m}*.*However, in many scientific and engineering
applications, it is desirable to have a real-time solution for the VI and CP problems.

Thus, at present, for solving the VI and CP problems, many researchers employ the neural network method which is a promising way to overcome this problem.

Neural networks for optimization were first introduced in the 1980s by Hopfield and Tank [20,38]. Since then, neural networks have been applied to various optimiza- tion problems, including linear programming, nonlinear programming, variational inequalities, and linear and nonlinear complementarity problems; see [5,9–11,17, 21,22,24,28,40–44]. The main idea of the neural network approach for optimiza- tion is to construct a nonnegative energy function and establish a dynamic system that represents an artificial neural network. The dynamic system is usually in the form of first order ordinary differential equations. Furthermore, it is expected that the dynamic system will approach its static state (or an equilibrium point), which corresponds to the solution for the underlying optimization problem, starting from an initial point. In addition, neural networks for solving optimization problems are hardware-implementable; that is, the neural networks can be implemented by using integrated circuits. In this paper, we focus on neural network approach to the SOCCVI problem. Our neural networks will be aimed to solve the system (5) whose solutions are candidates of SOCCVI problem (1).

The rest of this paper is organized as follows. Section2introduces some prelim- inaries. In Sect.3, the first neural network based on the Fischer-Burmeister function is proposed and studied. In Sect.4, we show that the KKT system (5) is equivalent to a nonlinear projection formulation. Then, the model of neural network for solv- ing the projection formulation is introduced and its stability is analyzed. In Sect.5, illustrative examples are discussed. Section6gives the conclusion of this paper.

**2 Preliminaries**

In this section, we recall some preliminary results that will be used later and back- ground materials of ordinary differential equations that will play an important role in the subsequent analysis. We begin with some concepts for a nonlinear mapping.

**Definition 2.1 Let F***= (F*1*, . . . , F*_{n}*)** ^{T}* : R

*→ R*

^{n}

^{n}*. Then, the mapping F is said to*be

(a) monotone if

*F (x) − F (y), x − y ≥ 0, ∀x, y ∈ R*^{n}*.*
(b) strictly monotone if

*F (x) − F (y), x − y > 0, ∀x, y ∈ R*^{n}*.*
*(c) strongly monotone with constant η > 0 if*

*F (x) − F (y), x − y ≥ ηx − y*^{2}*,* *∀x, y ∈ R*^{n}*.*
*(d) F is said to be Lipschitz continuous with constant γ if*

*F (x) − F (y) ≤ γ x − y, ∀x, y ∈ R*^{n}*.*

* Definition 2.2 Let X be a closed convex set in*R

^{n}*. Then, for each x*∈ R

^{n}*,*there is

*a unique point y∈ X such that x − y ≤ x − z, ∀z ∈ X. Here y is known as the*

*projection of x onto the set X with respect to Euclidean norm, that is,*

*y= P**X**(x)*= arg min

*z**∈X**x − z.*

*The projection function P**X**(x)*has the following property, called projection theo-
rem [2], which is useful in our subsequent analysis.

* Property 2.1 Let X be a nonempty closed convex subset of*R

^{n}*. Then, for each z*∈ R

^{n}*, P*

*X*

*(z)is the unique vector¯z ∈ X such that (y − ¯z)*

^{T}*(z− ¯z) ≤ 0, ∀y ∈ X.*

Next, we recall some materials about first order differential equations (ODE):

*˙w(t) = H(w(t)),* *w(t*0*)= w*0∈ R^{n}*,* (6)
*where H*: R* ^{n}*→ R

*is a mapping. We also introduce three kinds of stability that will be discussed later. These materials can be found in usual ODE textbooks, e.g. [31].*

^{n}**Definition 2.3 A point w**^{∗}*= w(t*^{∗}*)*is called an equilibrium point or a steady state
of the dynamic system (6) if H (w^{∗}*)= 0. If there is a neighborhood *^{∗}⊆ R^{n}*of w*^{∗}
*such that H (w*^{∗}*)= 0 and H(w)
= 0 ∀w ∈ *^{∗}*\ {w*^{∗}*}, then w*^{∗}is called an isolated
equilibrium point.

* Lemma 2.1 Assume that H*: R

*→ R*

^{n}

^{n}*is a continuous mapping. Then, for any t*0

*>*0

*and w*0∈ R

^{n}*, there exists a local solution w(t) for (6) with t∈ [t*0

*, τ )for some τ > t*0

*.*

*If, in addition, H is locally Lipschitz continuous at w*0

*, then the solution is unique; if*

*His Lipschitz continuous in*R

^{n}*, then τ can be extended to*∞.

If a local solution defined on*[t*0*, τ )*cannot be extended to a local solution on a
larger interval*[t*0*, τ*_{1}*), τ*1*> τ,*then it is called a maximal solution, and the interval
*[t*0*, τ )*is the maximal interval of existence. Clearly, any local solution has an exten-
sion to a maximal one. We denote*[t*0*, τ (w*_{0}*))*by the maximal interval of existence
*associated with w*0.

* Lemma 2.2 Assume that H*: R

*→ R*

^{n}

^{n}*is continuous. If w(t) with t∈ [t*0

*, τ (w*

_{0}

*))is*

*a maximal solution and τ (w*0

*) <∞, then lim*

*t*

*↑τ(w*0

*)*

*w(t) = ∞.*

**Definition 2.4 (Lyapunov Stability) Let w(t) be a solution for (6). An isolated equi-***librium point w*^{∗}*is Lyapunov stable if for any w*0*= w(t*0*)and any ε > 0, there exists*
*a δ > 0 such that ifw(t*0*)− w*^{∗}* < δ, then w(t) − w*^{∗}* < ε for all t ≥ t*0.

**Definition 2.5 (Asymptotic Stability) An isolated equilibrium point w**^{∗}is said to be
asymptotic stable if in addition to being Lyapunov stable, it has the property that if

*w(t*0*)− w*^{∗}* < δ, then w(t) → w*^{∗}*as t*→ ∞.

* Definition 2.6 (Lyapunov function) Let *⊆ R

*be an open neighborhood of*

^{n}*¯w.*

*A continuously differentiable function V*: R* ^{n}*→ R is said to be a Lyapunov function
at the state

*¯w over the set for (*6) if

*V (¯w) = 0, V (w) > 0, ∀w ∈ \ { ¯w},*

*˙V (w) ≤ 0, ∀w ∈ \ { ¯w}.* (7)

**Lemma 2.3**

*(a) An isolated equilibrium point w*^{∗} *is Lyapunov stable if there exists a Lyapunov*
*function over some neighborhood *^{∗}*of w*^{∗}.

*(b) An isolated equilibrium point w*^{∗} *is asymptotically stable if there exists a Lya-*
*punov function over some neighborhood *^{∗} *of w*^{∗} *such that ˙V (w) <0,* *∀w ∈*

^{∗}*\ {w*^{∗}}.

**Definition 2.7 (Exponential Stability) An isolated equilibrium point w**^{∗}is exponen-
*tially stable if there exists a δ > 0 such that arbitrary point w(t) of (6) with the initial*
*condition w(t*0*)= w*0and*w(t*0*)− w*^{∗}* < δ is well defined on [t*0*,+∞) and satisfies*

*w(t) − w*^{∗}* ≤ ce*^{−ωt}*w(t*0*)− w*^{∗}* ∀t ≥ t*0*,*
*where c > 0 and ω > 0 are constants independent of the initial point.*

**3 Neural network model based on smoothed Fischer-Burmeister function**
The smoothed Fischer-Burmeister function over the second-order cone defined be-
low is used to construct a merit function by which the KKT system of SOCCVI is
reformulated as an unconstrained smooth minimization problem. Furthermore, based

on the minimization problem, we propose a neural network and study its stability in this section.

*For any a= (a*1*; a*2*), b= (b*1*; b*2*)*∈ R × R^{n}^{−1}*,*we define their Jordan product as
*a· b = (a*^{T}*b; b*1*a*_{2}*+ a*1*b*_{2}*).*

*We denote a*^{2}*= a · a and |a| =*√

*a*^{2}*, where for any b∈ K** ^{n}*,√

*b*is the unique vector
in*K*^{n}*such that b*=√

*b*·√
*b.*

* Definition 3.1 A function φ*: R

*× R*

^{n}*→ R*

^{n}*is called an SOC-complementarity function if it satisfies*

^{n}*φ (a, b)= 0 ⇐⇒ a · b = 0, a ∈ K*^{n}*, b∈ K*^{n}*.*

A popular SOC-complementarity function is the Fischer-Burmeister function, which is semismooth [33] and defined as

*φ*_{FB}*(a, b)*=

*a*^{2}*+ b*^{2}*1/2*

*− (a + b).*

Then the smoothed Fischer-Burmeister function is given by

*φ*_{FB}^{ε}*(a, b)*=

*a*^{2}*+ b*^{2}*+ ε*^{2}*e*

*1/2*

*− (a + b)* (8)

*with ε*∈ R_{+}*and e= (1, 0, . . . , 0)** ^{T}* ∈ R

*.*

^{n}*The following lemma gives the gradient of φ*_{FB}* ^{ε}* . Since the proofs can be found in
[14,33,37], we here omit them.

**Lemma 3.1 Let φ**_{FB}^{ε}*be defined as in (8) and ε
= 0. Then, φ*_{FB}^{ε}*is continuously differ-*
*entiable everywhere and*

∇*ε**φ*_{FB}^{ε}*(a, b)= e*^{T}*L*^{−1}_{z}*L**εe**,* ∇*a**φ*_{FB}^{ε}*(a, b)= L*^{−1}*z* *L**a**− I,*

∇*b**φ*_{FB}^{ε}*(a, b)= L*^{−1}*z* *L**b**− I,*

*where z*=

*a*^{2}*+ b*^{2}*+ ε*^{2}*e**1/2*

*, I* *is identity mapping and L**a*=_{a}_{1} _{a}^{T}

2

*a*2*a*1*I**n*−1

*for a*=
*(a*_{1}*; a*2*)*∈ R × R^{n}^{−1}.

Using Definition3.1and KKT condition described in [37], we can see that the KKT system (5) is equivalent to the following unconstrained smooth minimization problem:

*min (w)*:=1

2*S(w)*^{2}*.* (9)

*Here (w), w= (ε, x, μ, λ) ∈ R*^{1}^{+n+l+m}*, is a merit function, and S(w) is defined*
by

*S(w)*=

⎛

⎜⎜

⎜⎜

⎜⎜

⎜⎝

*ε*
*L(x, μ, λ)*

*−h(x)*
*φ*^{ε}

FB*(−g**m*1*(x), λ*_{m}_{1}*)*
*...*

*φ*^{ε}

FB*(−g**m**p**(x), λ*_{m}_{p}*)*

⎞

⎟⎟

⎟⎟

⎟⎟

⎟⎠
*,*

*with g**m**i**(x), λ*_{m}* _{i}*∈ R

^{m}

^{i}*.In other words, (w) given in (9) is a smooth merit function*for the KKT system (5).

Based on the above smooth minimization problem (9), it is natural to propose the first neural network for solving the KKT system (5) of SOCCVI problem:

*dw(t )*

*dt* *= −ρ∇(w(t)),* *w(t*_{0}*)= w*0*,* (10)
*where ρ > 0 is a scaling factor.*

*Remark 3.1 In fact, we can also adopt another merit function which is based on the*
*FB function without the element ε. That is, we can define*

*S(x, μ, λ)*=

⎛

⎜⎜

⎜⎜

⎜⎝

*L(x, μ, λ)*

*−h(x)*
*φ*_{FB}*(−g**m*1*(x), λ*_{m}_{1}*)*

*...*

*φ*_{FB}*(−g**m**p**(x), λ*_{m}_{p}*)*

⎞

⎟⎟

⎟⎟

⎟⎠

*.* (11)

Then, the neural network model (10) could be obtained as well because*φ*FB^{2} is
smooth [7]. However, it is observed that the gradient mapping*∇ has more com-*
*plicated formulas because (−g**m**i**(x))*^{2}*+ λ*^{2}_{m}* _{i}* may lie on the boundary of SOC, or
interior of SOC, see [7,33], which will cost more expensive numerical computa-

*tions. Thus, the one dimensional parameter ε in use not only has no influence on the*main result, but also will simplify the computational work.

To discuss properties of the neural network model (10), we make the following
assumption which is used to avoid the singularity of*∇S(w), see [37].*

**Assumption 3.1**

*(a) the gradients{∇h**j**(x)|j = 1, . . . , l} ∪ {∇g**i**(x)|i = 1, . . . , m} are linear indepen-*
*dent.*

(b) ∇*x**L(x, μ, λ)is positive definite on the null space of the gradients{∇h**j**(x)|j =*
*1, . . . , l*}.

When SOCCVI problem corresponds to the KKT conditions of a convex second- order cone program (CSOCP) problem as (4) where both h and g are linear, the above

Assumption3.1(b) is indeed equivalent to the well-used condition∇^{2}*f (x)*is positive
definite, e.g. [42, Corollary 1].

* Proposition 3.1 Let *: R

^{1}

*→ R+*

^{+n+l+m}*be defined as in (9). Then, (w)≥ 0 for*

*w= (ε, x, μ, λ) ∈ R*

^{1}

^{+n+l+m}*and (w)= 0 if and only if (x, μ, λ) solves the KKT*

*system (5).*

*Proof The proof is straightforward.*

* Proposition 3.2 Let *: R

^{1}

*→ R+*

^{+n+l+m}*be defined as in (9). Then, the following*

*results hold.*

*(a) The function is continuously differentiable everywhere with*

*∇(w) = ∇S(w)S(w),*
*where*

*∇S(w) =*

⎡

⎢⎢

⎣

1 0 0 diag{∇*ε**φ*^{ε}

FB*(−g**m**i**(x), λ**m**i**)*}^{p}_{i}_{=1}
0 ∇*x**L(x, μ, λ)*^{T}*−∇h(x) −∇g(x)diag{∇**g*_{mi}*φ*^{ε}

FB*(−g**m**i**(x), λ*_{m}_{i}*)*}^{p}_{i}_{=1}

0 *∇h(x)** ^{T}* 0 0

0 *∇g(x)** ^{T}* 0 diag{∇

*λ*

_{mi}*φ*

^{ε}FB*(−g**m*_{i}*(x), λ**m*_{i}*)*}^{p}_{i}_{=1}

⎤

⎥⎥

*⎦ .*

*(b) Suppose that Assumption*3.1*holds. Then,* *∇S(w) is nonsingular for any w ∈*
R^{1}^{+n+l+m}*. Moreover, if (0, x, μ, λ)*∈ R^{1}^{+n+l+m}*is a stationary point of , then*
*(x, μ, λ)*∈ R^{n}^{+l+m}*is a KKT triple of the SOCCVI problem.*

*(c) (w(t)) is nonincreasing with respect to t .*

*Proof Part(a) follows from the chain rule. For part(b), we know that∇S(w) is non-*
singular if and only if the following matrix

⎡

⎣∇*x**L(x, μ, λ)*^{T}*−∇h(x) −∇g(x)diag{∇**g*_{mi}*φ*_{FB}^{ε}*(−g**m*_{i}*(x), λ**m*_{i}*)*}^{p}_{i}_{=1}

*∇h(x)** ^{T}* 0 0

*∇g(x)** ^{T}* 0 diag{∇

*λ*

_{mi}*φ*

_{FB}

^{ε}*(−g*

*m*

*i*

*(x), λ*

_{m}

_{i}*)*}

^{p}

_{i}_{=1}

⎤

⎦

is nonsingular. In fact, from [37, Theorem 3.1] and [37, Proposition 4.1], the above
*matrix is nonsingular and (x, μ, λ)*∈ R^{n}* ^{+l+m}* is a KKT triple of the SOCCVI prob-

*lem if (0, x, μ, λ)*∈ R

^{1}

^{+n+l+m}*is a stationary point of . It remains to show part(c).*

*By the definition of (w) and (10), it is not difficult to compute*
*d(w(t ))*

*dt* *= ∇(w(t))*^{T}*dw(t )*

*dt* *= −ρ∇(w(t))*^{2}*≤ 0.* (12)
*Therefore, (w(t)) is a monotonically decreasing function with respect to t .*
Now, we are ready to analyze the behavior of the solution trajectory of (10) and
establish properties of three kinds of stability for an isolated equilibrium point.

**Proposition 3.3**

*(a) If (x, μ, λ)*∈ R^{n}^{+l+m}*is a KKT triple of SOCCVI problem, then (0, x, μ, λ)*∈
R^{1}^{+n+l+m}*is an equilibrium point of (10).*

*(b) If Assumption*3.1*holds and (0, x, μ, λ)*∈ R^{1}^{+n+l+m}*is an equilibrium point of*
(10), then (x, μ, λ)∈ R^{n}^{+l+m}*is a KKT triple of SOCCVI problem.*

*Proof (a) From Proposition*3.1*and (x, μ, λ)*∈ R^{n}* ^{+l+m}*being a KKT triple of SOC-

*CVI problem, it is clear that S(0, x, μ, λ)= 0. Hence, ∇(0, x, μ, λ) = 0. Besides,*by Proposition3.2, we know that if ε

*= 0, then ∇(ε, x, μ, λ) = 0. This shows that*

*(0, x, μ, λ) is an equilibrium point of (10).*

*(b) It follows from (0, x, μ, λ)*∈ R^{1}* ^{+n+l+m}*being an equilibrium point of (10) that

*∇(0, x, μ, λ) = 0. In other words, (0, x, μ, λ) is the stationary point of . Then,*
the result is a direct consequence of Proposition3.2(b).
**Proposition 3.4**

*(a) For any initial state w*0*= w(t*0*), there exists exactly one maximal solution w(t)*
*with t∈ [t*0*, τ (w*_{0}*))for the neural network (10).*

*(b) If the level set* *L(w*0*)= {w ∈ R*^{1}^{+n+l+m}*|(w) ≤ (w*0*)} is bounded, then*
*τ (w*0*)*= +∞.

*Proof (a) Since S is continuous differentiable,∇S is continuous, and therefore, ∇S*
*is bounded on a local compact neighborhood of w. That means∇(w) is locally*
Lipschitz continuous. Thus, applying Lemma2.1leads to the desired result.

(b) This proof is similar to the proof of Case(i) in [5, Proposition 4.2].
*Remark 3.2 We wish to obtain the result that the level sets*

*L(, γ ) := {w ∈ R*^{1}^{+n+l+m}*| (w) ≤ γ }*

*are bounded for all γ* ∈ R. However, we are not able to complete the argument. We
*suspect that there needs more subtle properties of F , h and g to finish it.*

Next, we investigate the convergence of the solution trajectory of neural net- work (10).

**Theorem 3.1**

*(a) Let w(t) with t∈ [t*0*, τ (w*_{0}*))be the unique maximal solution to (10). If τ (w*0*)*=
*+∞ and {w(t)} is bounded, then lim**t*→∞*∇(w(t)) = 0.*

*(b) If Assumption*3.1*holds and (ε, x, μ, λ)*∈ R^{1}^{+n+l+m}*is the accumulation point of*
*the trajectory w(t ), then (x, μ, λ)*∈ R^{n}^{+l+m}*is a KKT triple of SOCCVI problem.*

*Proof With Proposition*3.2(b) and (c) and Proposition3.3, the arguments are exactly
the same as those for [28, Corollary 4.3]. Thus, we omit them.
**Theorem 3.2 Let w**^{∗} *be an isolated equilibrium point of the neural network (10).*

*Then the following results hold.*

*(a) w*^{∗}*is asymptotically stable.*

*(b) If Assumption*3.1*holds, then it is exponentially stable.*

*Proof Since w*^{∗}is an isolated equilibrium point of (10), there exists a neighborhood

^{∗}⊆ R^{1}^{+n+l+m}*of w*^{∗}such that

*∇(w*^{∗}*)= 0 and ∇(w)
= 0 ∀w ∈ *^{∗}*\ {w*^{∗}*}.*

*Next, we argue that (w) is indeed a Lyapunov function at x*^{∗} *over the set *^{∗}for
(10) by showing that the conditions in (7) are satisfied. First, notice that (w)≥ 0.

Suppose that there is an *¯w ∈ *^{∗}*\ {w*^{∗}*} such that ( ¯w) = 0. Then, we can easily*
obtain that*∇( ¯w) = 0, i.e., ¯w is also an equilibrium point of (10), which clearly*
*contradicts the assumption that w*^{∗}*is an isolated equilibrium point in *^{∗}. Thus, we
*prove that (w) > 0 for any w∈ *^{∗}*\ {w*^{∗}}. This together with (12) shows that the
condition in (7) are satisfied. Because w^{∗}is isolated, from (12), we have

*d(w(t ))*

*dt* *<0,* *∀w(t) ∈ *^{∗}*\ {w*^{∗}*}.*

*This implies that w*^{∗}is asymptotically stable. Furthermore, if Assumption3.1holds,
we can obtain that*∇S is nonsingular. In addition, we have*

*S(w)= S(w*^{∗}*)+ ∇S(w*^{∗}*)(w− w*^{∗}*)+ o(w − w*^{∗}*), ∀w ∈ *^{∗}*\ {w*^{∗}*}.* (13)
From *S(w(t)) being a monotonically decreasing function with respect to t and*
(13), we can deduce that

*w(t) − w*^{∗}* ≤ (∇S(w*^{∗}*))*^{−1}*S(w(t)) − S(w*^{∗}*) + o(w(t) − w*^{∗}*)*

*≤ (∇S(w*^{∗}*))*^{−1}*S(w(t*0*))− S(w*^{∗}*) + o(w(t) − w*^{∗}*)*

*≤ (∇S(w*^{∗}*))*^{−1}

*(∇S(w*^{∗}*))w(t*0*)− w*^{∗}* + o(w(t*0*)− w*^{∗}*)*
*+ o(w(t) − w*^{∗}*).*

That is,

*w(t) − w*^{∗}* − o(w(t) − w*^{∗}*) ≤ (∇S(w*^{∗}*))*^{−1}

*(∇S(w*^{∗}*))w(t*0*)− w*^{∗}
*+ o(w(t*0*)− w*^{∗}*)*

*.*

The above inequality implies that the neural network (10) is also exponentially sta-

ble.

**4 Neural network model based on projection function**

In this section, we present that the KKT triple of SOCCVI problem is equivalent to the solution of a projection formulation. Based on this, we introduce another neural network model for solving the projection formulation and analyze the stability con- ditions and convergence.

*Define the function U*: R^{n}* ^{+l+m}*→ R

^{n}

^{+l+m}*and vector w in the following form:*

*U (w)*=

⎛

⎝*L(x, μ, λ)*

*−h(x)*

*−g(x)*

⎞

*⎠ , w =*

⎛

⎝*x*
*μ*
*λ*

⎞

*⎠ ,* (14)

*where L(x, μ, λ)= F (x) + ∇h(x)μ + ∇g(x)λ is the Lagrange function. To avoid*
*confusion, we emphasize that, for any w*∈ R^{n}* ^{+l+m}*, we have

*w*_{i}*∈ R, if 1 ≤ i ≤ n + l,*

*w**i* ∈ R^{m}^{i−(n+l)}*,* *if n+ l + 1 ≤ i ≤ n + l + p.*

Then, we may write (14) as

*U*_{i}*= (U(w))**i**= (L(x, μ, λ))**i**,* *w*_{i}*= x**i**, i= 1, . . . , n,*
*U**n**+j**= (U(w))**n**+j**= −h**j**(x),* *w**n**+j**= μ**j**, j= 1, . . . , l,*

*U*_{n}_{+l+k}*= (U(w))**n**+l+k**= −g**k**(x)*∈ R^{m}^{k}*,* *w*_{n}_{+l+k}*= λ**k*∈ R^{m}^{k}*, k= 1, . . . , p,*

*p*
*k*=1

*m**k**= m.*

With this, the KKT conditions (5) can be recast as
*U*_{i}*= 0, i = 1, 2, . . . , n, n + 1, . . . , n + l,*

*U**J**, w**J** = 0, U**J**= (U**n**+l+1**, U**n**+l+2**, . . . , U**n**+l+p**)*^{T}*∈ K,* (15)
*w**J**= (w**n**+l+1**, w**n**+l+2**, . . . , w**n**+l+p**)*^{T}*∈ K.*

*Thus, (x*^{∗}*, μ*^{∗}*, λ*^{∗}*)* is a KKT triple for (1) if and only if (x^{∗}*, μ*^{∗}*, λ*^{∗}*)*is a solution
to (15).

It is well known that the nonlinear complementarity problem, which is denoted by
*NCP(F, K) and to find an x*∈ R* ^{n}*such that

*x∈ K, F (x) ∈ K and F (x), x = 0*

*where K is a closed convex set of*R^{n}*, is equivalent to the following VI(F, K) prob-*
*lem: finding an x∈ K such that*

*F (x), y − x ≥ 0 ∀y ∈ K.*

*Furthermore, if K*= R^{n}*, then NCP(F, K) becomes the system of nonlinear equations*
*F (x)= 0.*

Based on the above, solution of (15) is equivalent to solution of the following VI
*problem: find w*∈ K such that

*U(w), v − w ≥ 0, ∀v ∈ K,* (16)

whereK = R^{n}^{+l}*× K. In addition, by applying the Property*2.1, its solution is equiv-
alent to solution of below projection formulation

*P*_{K}*(w− U(w)) = w with K = R*^{n}^{+l}*× K,* (17)
*where function U and vector w are defined in (14). Now, according to (17), we give*
the following neural network:

*dw*

*dt* *= ρ{P*_{K}*(w− U(w)) − w},* (18)

*where ρ > 0. Note thatK is a closed and convex set. For any w ∈ R*^{n}^{+l+m}*, P*_{K}means
*P*_{K}*(w)= [P*_{K}*(w*1*), P*_{K}*(w*2*), . . . , P*_{K}*(w**n**+l**), P*_{K}*(w**n**+l+1**), P*_{K}*(w**n**+l+2**), . . . ,*

*P*_{K}*(w*_{n}_{+l+p}*)],*
where

*P*_{K}*(w**i**)= w**i**,* *i= 1, . . . , n + l,*

*P*_{K}*(w*_{n}_{+l+j}*)= [λ*1*(w*_{n}_{+l+j}*)*]_{+}*· u*^{(1)}_{w}_{n}_{+l+j}*+ [λ*2*(w*_{n}_{+l+j}*)*]_{+}*· u*^{(2)}_{w}_{n}_{+l+j}*,* *j= 1, . . . , p.*

*Here, for the sake of simplicity, we denote the vector w**n**+l+j* *by v for the moment,*
and[·]_{+} *is the scalar projection, λ*1*(v), λ*2*(v)and u*^{(1)}*v* *, u*^{(2)}*v* are the spectral values
*and the associated spectral vectors of v= (v*1*; v*2*)*∈ R × R^{m}^{j}^{−1}, respectively, given

by ⎧

⎨

⎩

*λ**i**(v)= v*1*+ (−1)*^{i}*v*2*,*
*u*^{(i)}* _{v}* =

^{1}

_{2}

*(1, (−1)*

^{i v}

_{v}^{2}

_{2}

_{}

*),*

*for i= 1, 2, see [7,*33].

The dynamic system described by (18) can be recognized as a recurrent neural network with a single-layer structure. To analyze the stability conditions of (18), we need the following lemmas and proposition.

**Lemma 4.1 If the gradient of L(x, μ, λ) is positive semi-definite (respectively, pos-***itive definite), then the gradient of U in (14) is positive semi-definite (respectively,*
*positive definite).*

*Proof Since we have*

*∇U(x, μ, λ) =*

⎡

⎣∇*x**L*^{T}*(x, μ, λ)* *−∇h(x) −∇g(x)*

∇^{T}*h(x)* 0 0

∇^{T}*g(x)* 0 0

⎤

*⎦ ,*

*for any nonzero vector d= (p*^{T}*, q*^{T}*, r*^{T}*)** ^{T}* ∈ R

^{n}*, we can obtain that*

^{+l+m}*d*^{T}*∇U(x, μ, λ)d =*

*p*^{T}*q*^{T}*r** ^{T}*⎡

⎣∇*x**L*^{T}*(x, μ, λ)* *−∇h(x) −∇g(x)*

∇^{T}*h(x)* 0 0

∇^{T}*g(x)* 0 0

⎤

⎦

⎛

⎝*p*
*q*
*r*

⎞

⎠

*= p** ^{T}*∇

*x*

*L(x, μ, λ)p.*

This leads to the desired results.

* Proposition 4.1 For any initial point w*0

*= (x*0

*, μ*

_{0}

*, λ*

_{0}

*)with λ*0

*:= λ(t*0

*)∈ K, there*

*exist a unique solution w(t )= (x(t), μ(t), λ(t)) for neural network (18), Moreover,*

*λ(t )∈ K.*

*Proof For simplicity, we assumeK = K** ^{m}*. The analysis can be carried over to the
general case where

*K is the Cartesian product of second-order cones. Since F, h, g*are continuous differentiable, the function

*F (w):= P*K*(w− U(w)) − w with K = R*^{n}^{+l}*× K** ^{m}* (19)
is semi-smooth and Lipschitz continuous. Thus, there exists a unique solution

*w(t )= (x(t), μ(t), λ(t)) for neural network (18). Therefore, it remains to show that*

*λ(t )∈ K*

^{m}*. For convenience, we denote λ(t):= (λ*1

*(t ), λ*

_{2}

*(t ))*∈ R × R

^{m}^{−1}. To com-

*plete the proof, we need to verify two things: (i) λ*1

*(t )≥ 0 and (ii) λ*2

*(t ) ≤ λ*1

*(t ).*

First, from (18), we have
*dλ*

*dt* *+ ρλ(t) = ρP*_{K}^{m}*(λ+ g(x)).*

The solution of the above first-order ordinary differential equation is
*λ(t )= e*^{−ρ(t−t}^{0}^{)}*λ(t*0*)+ ρe*^{−ρt}

*t*
*t*0

*ρe*^{ρs}*P*_{K}^{m}*(λ+ g(x))ds.* (20)

*If we let λ(t*0*):= (λ*1*(t*0*), λ*2*(t*0*))*∈ R × R^{m}^{−1}*and denote P*_{K}^{m}*(λ+ g(x)) as z(t*0*)*:=

*(z*1*(t*0*), z*2*(t*0*)), then (20) leads to*

*λ*_{1}*(t )= e*^{−ρ(t−t}^{0}^{)}*λ*_{1}*(t*_{0}*)+ ρe*^{−ρt}

_{t}

*t*0

*ρe*^{ρs}*z*_{1}*(s)ds,* (21)

*λ*_{2}*(t )= e*^{−ρ(t−t}^{0}^{)}*λ*_{2}*(t*_{0}*)+ ρe*^{−ρt}

_{t}

*t*_{0}

*ρe*^{ρs}*z*_{2}*(s)ds.* (22)
*Due to both λ(t*0*)and z(t) belong to* *K*^{m}*, there have λ*1*(t*0*)≥ 0, λ*2*(t*0*) ≤ λ*1*(t*0*)*
and*z*2*(t ) ≤ z*1*(t ). Therefore, λ*1*(t )*≥ 0 since both terms in the right-hand side of
(21) are nonnegative. In addition,

*λ*2*(t ) ≤ e*^{−ρ(t−t}^{0}^{)}*λ*2*(t*0*) + ρe*^{−ρt}

_{t}

*t*0

*ρe*^{ρs}*z*2*(s)ds*

*≤ e*^{−ρ(t−t}^{0}^{)}*λ*1*(t*0*)+ ρe*^{−ρt}

_{t}

*t*0

*ρe*^{ρs}*z*1*(s)ds*

*= λ*1*(t ),*

*which implies that λ(t)∈ K** ^{m}*.

**Lemma 4.2 Let U (w), F (w) be defined as in (14) and (19), respectively. Suppose***w*^{∗}*= (x*^{∗}*, μ*^{∗}*, λ*^{∗}*)is an equilibrium point of neural network (18) with (x*^{∗}*, μ*^{∗}*, λ*^{∗}*)*
*being an KKT triple of SOCCVI problem. Then, the following inequality holds:*

*(F (w)+ w − w*^{∗}*)*^{T}*(−F (w) − U(w)) ≥ 0.* (23)
*Proof Notice that*

*(F (w)+ w − w*^{∗}*)*^{T}*(−F (w) − U(w))*

*= [−w + P*_{K}*(w− U(w)) + w − w*^{∗}]^{T}*[w − P*_{K}*(w− U(w)) − U(w)]*

*= [−w*^{∗}*+ P*_{K}*(w− U(w))]*^{T}*[w − P*_{K}*(w− U(w)) − U(w)]*

*= −[w*^{∗}*− P*_{K}*(w− U(w))]*^{T}*[w − U(w) − P*_{K}*(w− U(w))].*

*Since w*^{∗}∈ K, applying Property2.1gives

*[w*^{∗}*− P*_{K}*(w− U(w))]*^{T}*[w − U(w) − P*_{K}*(w− U(w))] ≤ 0.*

Thus, we have

*(F (w)+ w − w*^{∗}*)*^{T}*(−F (w) − U(w)) ≥ 0.*

This completes the proof.

We now show the stability and convergence issues regarding neural network (18).

* Theorem 4.1 If*∇

*x*

*L(w)is positive semi-definite (respectively, positive definite), the*

*solution of neural network (18) with initial point w*0

*= (x*0

*, μ*0

*, λ*0

*)where λ*0

*∈ K is*

*Lyapunov stable (respectively, asymptotically stable). Moreover, the solution trajec-*

*tory of neural network (18) is extendable to the global existence.*

*Proof Again, for simplicity, we assumeK = K** ^{m}*. From Proposition4.1, there exists

*a unique solution w(t)= (x(t), μ(t), λ(t)) for neural network (*18) and λ(t)

*∈ K*

*.*

^{m}*Let w*

^{∗}

*= (x*

^{∗}

*, μ*

^{∗}

*, λ*

^{∗}

*)*be an equilibrium point of neural network (18). We define a Lyapunov function as below:

*V (w):= V (x, μ, λ) := −U(w)*^{T}*F (w)*−1

2*F (w)*^{2}+1

2*w − w*^{∗}^{2}*.* (24)
From [12, Theorem 3.2], we know that V is continuously differentiable with

*∇V (w) = U(w) − [∇U(w) − I]F (w) + (w − w*^{∗}*).*

*It is also trivial that V (w*^{∗}*)*= 0. Then, we have

*dV (w(t ))*

*dt* *= ∇V (w(t))*^{T}*dw*
*dt*

*= {U(w) − [∇U(w) − I]F (w) + (w − w*^{∗}*)*}^{T}*ρF (w)*

*= ρ{[U(w) + (w − w*^{∗}*)*]^{T}*F (w)+ F (w)*^{2}*− F (w)*^{T}*∇U(w)F (w)}.*

Inequality (23) in Lemma4.2implies

*[U(w) + (w − w*^{∗}*)]*^{T}*F (w)≤ −U(w)*^{T}*(w− w*^{∗}*)− F (w)*^{2}*,*
which yields

*dV (w(t ))*
*dt*

*≤ ρ{−U(w)*^{T}*(w− w*^{∗}*)− F (w)*^{T}*∇U(w)F (w)}*

*= ρ{−U(w*^{∗}*)*^{T}*(w− w*^{∗}*)− (U(w) − U(w*^{∗}*))*^{T}*(w− w*^{∗}*)*

*− F (w)*^{T}*∇U(w)F (w)}.* (25)

*Note that w*^{∗} is the solution of the variational inequality (16). Since w∈ K, we
therefore obtain *−U(w*^{∗}*)*^{T}*(w− w*^{∗}*)≤ 0. Because U(w) is continuous differen-*
tiable and *∇U(w) is positive semi-definite, by [32, Theorem 5.4.3], we obtain*
*that U (w) is monotone. Hence, we have−(U(w) − U(w*^{∗}*))*^{T}*(w− w*^{∗}*)*≤ 0 and

*−F (w)*^{T}*∇U(w)F (w) ≤ 0. The above discussions lead to*^{dV (w(t ))}* _{dt}* ≤ 0. Also, by [32,
Theorem 5.4.3], we know that if

*∇U(w) is positive definite, then U(w) is strictly*monotone, which implies

^{dV (w(t ))}

_{dt}*<*0 in this case.

*In order to obtain V (w) is a Lyapunov function and w*^{∗} is Lyapunov stable, we
will show the following inequality:

*−U(w)*^{T}*F (w)≥ F (w)*^{2}*.* (26)
To see this, we first observe that

*F (w)*^{2}*+ U(w)*^{T}*F (w)= [w − P*K*(w− U(w))]*^{T}*[w − U(w) − P*K*(w− U(w))].*

*Since w*∈ K, applying Property2.1again, there holds

*[w − P*_{K}*(w− U(w))]*^{T}*[w − U(w) − P*_{K}*(w− U(w))] ≤ 0,*
which yields the desired inequality (26). By combining (24) and (26), we have

*V (w)*≥1

2*F (w)*^{2}+1

2*w − w*^{∗}^{2}*,*

*which says V (w) > 0 if w
= w*^{∗}*. Hence V (w) is indeed a Lyapunov function and*
*w*^{∗} is Lyapunov stable. Furthermore, if∇*x**L(w)is positive definite, we have w*^{∗} is
asymptotically stable. Moreover, it holds that

*V (w*_{0}*)≥ V (w) ≥*1

2*w − w*^{∗}^{2} *for t≥ t*0*,* (27)

*which tells us the solution trajectory w(t) is bounded. Hence, it can be extended to*

global existence.

**Theorem 4.2 Let w**^{∗}*= (x*^{∗}*, μ*^{∗}*, λ*^{∗}*)* *be an equilibrium point of (18). If* ∇*x**L(w)*
*is positive definite, the solution of neural network (18) with initial point w*0=
*(x*0*, μ*0*, λ*0*)where λ*0*∈ K is globally convergent to w*^{∗} *and has finite convergence*
*time.*

*Proof From (27), the level set*

*L(w*0*):= {w|V (w) ≤ V (w*0*)*}

is bounded. Then, the Invariant Set Theorem [15] implies the solution trajectory w(t)
*converges to θ as t→ +∞ where θ is the largest invariant set in*

=

*w∈ L(w*0*)*

*dV (w(t ))*

*dt* = 0

*.*

*We will show that dw/dt= 0 if and only if dV (w(t))/dt = 0 which yields that w(t)*
*converges globally to the equilibrium point w*^{∗}*= (x*^{∗}*, μ*^{∗}*, λ*^{∗}*). Suppose dw/dt* = 0,
*then it is clear that dV (w(t))/dt= ∇V (w)*^{T}*(dw/dt )= 0. Let ˆw = ( ˆx, ˆμ, ˆλ) ∈ *
*which says dV (ˆw(t))/dt = 0. From (*23), we know that

*dV (ˆw(t))/dt ≤ ρ{(−U( ˆw) − U(w*^{∗}*))*^{T}*(ˆw − w*^{∗}*)− F ( ˆw)*^{T}*∇U( ˆw)F ( ˆw)}.*

Both terms inside the big parenthesis are nonpositive as shown in Theorem4.1, so
*(U (ˆw) − U(w*^{∗}*))*^{T}*(ˆw − w*^{∗}*)= 0, F ( ˆw)*^{T}*∇U( ˆw)F ( ˆw) = 0. The condition ∇**x**L(w)*
being positive definite leads to*∇U( ˆw) being positive definite. Hence,*

*F (ˆw) = − ˆw + P*K*(ˆw − U( ˆw)) = 0,*

*which is equivalent to dˆw/dt = 0. From the above, w(t) converges globally to the*
*equilibrium point w*^{∗}*= (x*^{∗}*, μ*^{∗}*, λ*^{∗}*). Moreover, with Theorem*4.1and following the
same argument as in [42, Theorem 2], the neural network (18) has finite convergence

time.

**5 Simulations**

To demonstrate effectiveness of the proposed neural networks, some illustrative SOC-
CVI problems are tested. The numerical implementation is coded by Matlab 7.0 and
*the ordinary differential equation solver adopted is ode23, which uses Runge-Kutta*
(2), (3) formula. In the following tests, the parameter ρ in both neural networks is set
to be 1000.

*Example 5.1 Consider the SOCCVI problem (1)–(2) where*

*F (x)*=

⎛

⎜⎜

⎜⎜

⎜⎜

⎜⎜

⎜⎜

⎜⎜

⎜⎜

⎝

*2x*1*+ x*2+ 1
*x*1*+ 6x*2*− x*3− 2

*−x*2*+ 3x*3−^{6}_{5}*x*_{4}+ 3

−^{6}_{5}*x*3*+ 2x*4+^{1}_{2}*sin x*4*cos x*5*sin x*6+ 6

1

2*cos x*4*sin x*5*sin x*6*+ 2x*5−^{5}_{2}

−^{1}_{2}*cos x*4*cos x*5*cos x*6*+ 2x*6+^{1}_{4}*cos x*6*sin x*7*cos x*8+ 1

1

4*sin x*6*cos x*7*cos x*8*+ 4x*7− 2

−^{1}_{4}*sin x*6*sin x*7*sin x*8*+ 2x*8+^{1}_{2}

⎞

⎟⎟

⎟⎟

⎟⎟

⎟⎟

⎟⎟

⎟⎟

⎟⎟

⎠

and

*C= {x ∈ R*^{8}*: −g(x) = x ∈ K*^{3}*× K*^{3}*× K*^{2}*}.*

This problem has an approximate solution

*x*^{∗}*= (0.3820, 0.1148, −0.3644, 0.0000, 0.0000, 0.0000, 0.5000, −0.2500)*^{T}*.*
It can be verified that the Lagrangian function for this example is

*L(x, μ, λ)= F (x) − λ*
and the gradient of the Lagrangian function is

*∇L(x, μ, λ) =*

*∇F (x)*
*I*8×8

*,*

*where I is the identity mapping and∇F (x) means the gradient of F (x). We use the*
proposed neural networks with smoothed FB and projection functions, respectively,
to solve the problem whose trajectories are depicted in Figs.1and2. The simulation
*results show that both trajectories are globally convergent to x*^{∗} and the neural net-
*work with projection function converges to x*^{∗}quicker than that with smoothed FB
function.

*Example 5.2 [37, Example 5.1] We consider the following SOCCVI problem:*

1

2*Dx, y− x*

!

*≥ 0, ∀y ∈ C*

where

*C= {x ∈ R*^{n}*: Ax − a = 0, Bx − b 0},*

*Dis an n× n symmetric matrix, A and B are l × n and m × n matrices, respectively,*
*dis an n× 1 vector, a and b are l × 1 and m × 1 vectors with l + m ≤ n, respectively.*

**Fig. 1 Transient behavior of neural network with smoothed FB function in Example**5.1

**Fig. 2 Transient behavior of the neural network with projection function in Example**5.1

*In fact, we can determine the data a, b, A, B and D randomly. However, as in [37,*
Example 5.1], we set the data as follows:

*D= [D**ij*]*n**×n**,* *where D**ij*=

⎧⎨

⎩

*2,* *i= j,*
*1,* *|i − j| = 1,*
*0,* *otherwise,*

*A= [I**l**×l* 0*l**×(n−l)*]*l**×n**, B*= [0*m**×(n−m)* *I*_{m}* _{×m}*]

*m*

*×n*

*, a*= 0

*l*×1

*, b= (e*

*m*1

*, e*

_{m}_{2}

*, . . . ,*

*e*

_{m}

_{p}*)*

^{T}*, where e*

*m*

*i*

*= (1, 0, . . . , 0)*

*∈ R*

^{T}

^{m}

^{i}*and l+ m ≤ n. Clearly, A and B are full*

*row rank and rank([A*

^{T}*B*

^{T}*]) = l + m.*

*In the simulation, the parameters l, m, and n are set to be 3, 3, and 6, respec-*
*tively. The problem has an solution x*^{∗}*= (0, 0, 0, 0, 0, 0)** ^{T}*. It can be verified that the
Lagrangian function for this example is

*L(x, μ, λ)*=1

2*Dx+ A*^{T}*μ+ B*^{T}*λ.*

Note that∇*x**L(x, μ, λ)*is positive definite. We know from Theorems3.1and4.2that
both proposed neural networks are globally convergent to the KKT triple of the SOC-
CVI problem. Figures3and4depict the trajectories of Example5.2obtained using
the proposed neural networks. The simulation results show that both neural networks
are effective in the SOCCVI problem and the neural network with projection function
*converges to x*^{∗}quicker than that with smoothed FB function.

*Example 5.3 Consider the SOCCVI problem (1)–(2) where*

*F (x)*=

⎛

⎜⎜

⎜⎜

⎜⎜

⎝

*x*3*exp(x*1*x*3*)+ 6(x*1*+ x*2*)*
*6(x*1*+ x*2*)*+√^{2(2x}^{2}^{−x}^{3}^{)}

1*+(2x*2*−x*3*)*^{2}

*x*_{1}*exp(x*1*x*_{3}*)*−√ ^{2x}^{2}^{−x}^{3}

1*+(2x*2*−x*3*)*^{2}

*x*4

*x*5

⎞

⎟⎟

⎟⎟

⎟⎟

⎠

and

*C= {x ∈ R*^{5}*: h(x) = 0, −g(x) ∈ K*^{3}*× K*^{2}*},*
with

*h(x)= −62x*1^{3}*+ 58x*2*+ 167x*3^{3}*− 29x*3*− x*4*− 3x*5*+ 11,*

*g(x)*=

⎛

⎜⎜

⎜⎜

⎝

*−3x*_{1}^{3}*− 2x*2*+ x*3*− 5x*_{3}^{3}
*5x*_{1}^{3}*− 4x*2*+ 2x*3*− 10x*_{3}^{3}

*−x*3

*−x*4

*−3x*5

⎞

⎟⎟

⎟⎟

⎠*.*

*This problem has an approximate solution x*^{∗}*= (0.6287, 0.0039, −0.2717, 0.1761,*
*0.0587)** ^{T}*.

**Fig. 3 Transient behavior of neural network with smoothed FB function in Example**5.2

**Fig. 4 Transient behavior of neural network with projection function in Example**5.2

*Example 5.4 Consider the SOCCVI problem (1)–(2) where*

*F (x)*=

⎛

⎜⎜

⎝

*4x*1*− sin x*1*cos x*2+ 1

*− cos x*1*sin x*2*+ 6x*2+^{9}_{5}*x*3+ 2

9

5*x*_{2}*+ 8x*3+ 3
*2x*4+ 1

⎞

⎟⎟

⎠

and

*C*=

*x*∈ R^{4}*: h(x) =*

"

*x*_{1}^{2}−_{10}^{1}*x*2*x*3*+ x*3

*x*_{3}^{2}*+ x*4

#

*= 0, −g(x) =*

$*x*1

*x*2

%

*∈ K*^{2}

&

*.*

*This problem has an approximate solution x*^{∗} *= (0.2391, −0.2391, −0.0558,*

*−0.0031)** ^{T}*.

The neural network (10) based on smoothed FB function can solve Examples5.3–

5.4successfully, see Figs.5,6, whereas the neural network (18) based on projection
function fails to solve them. This is because that∇*x**L(x, μ, λ)*is not always positive
definite in Examples5.3and5.4. Hence, the neural network with projection function
is not effective in these two problems. To the contrast, though there is no guarantee
that the Assumption 3.1(b) holds, the neural network with smoothed FB function
is asymptotically stable from Theorem3.2. Figures5 and6 depict the trajectories
obtained using the neural network with the smoothed FB function for Examples5.3
and5.4, respectively. The simulation results show that each trajectory converges to
the desired isolated equilibrium point which is exactly the approximate solution of
Examples5.3and5.4, respectively.

*Example 5.5 Consider the nonlinear convex SOCP [23] given by*
min *exp(x*1*− x*3*)+ 3(2x*1*− x*2*)*^{4}+'

1*+ (3x*2*+ 5x*3*)*^{2}

*s.t.* *−g(x) =*

⎛

⎜⎜

⎜⎜

⎝

*4x*1*+ 6x*2*+ 3x*3− 1

*−x*1*+ 7x*2*− 5x*3+ 2
*x*1

*x*2

*x*_{3}

⎞

⎟⎟

⎟⎟

⎠*∈ K*^{2}*× K*^{3}*.*

*The approximate solution of this problem is x*^{∗}*= (0.2324, −0.07309, 0.2206)** ^{T}*. As
mentioned in Sect.1, the CSOCP in Example5.5can be transformed into an equiv-
alent SOCCVI problem. There are neural network models proposed for CSOCP
in [27]. We here try a different approach for it. In other words, we use the proposed
neural networks with the smoothed FB and projection functions, respectively, to solve
the problem whose trajectories are depicted in Figs.7 and8. From the simulation
results, we see that the neural network with smoothed FB function converges very
slowly and it is not clear whether it converges to the solution in finite time or not. In
view of these, to solve CSOCP, it seems better to apply the models introduced in [27]

directly instead of transforming CSOCP into SOCCVI problem.

**Fig. 5 Transient behavior of neural network with smoothed FB function in Example**5.3

**Fig. 6 Transient behavior of neural network with smoothed FB function in Example**5.4

**Fig. 7 Transient behavior of neural network with smoothed FB function in Example**5.5

**Fig. 8 Transient behavior of neural network with projection function in Example**5.5