A neural network based on the metric projector for solving SOCCVI problem

A neural network based on the metric projector for solving SOCCVI problem

Juhe Sun, Weichen Fu, Jan Harold Alcantara and Jein-Shan Chen

Abstract—We propose an efficient neural network for solving the second-order cone constrained variational inequality (SOC- CVI for short). The network is constructed using the Karush- Kuhn-Tucker (KKT) conditions of the variational inequality, which is used to recast the SOCCVI as a system of equations by using a smoothing function for the metric projection mapping to deal with the complementarity condition. Aside from standard stability results, we explore second-order sufficient conditions to obtain exponential stability. Specifically, we prove the non- singularity of the Jacobian of the KKT system based on the second-order sufficient condition and constraint nondegeneracy.

Finally, we present some numerical experiments illustrating the efficiency of the neural network in solving SOCCVI problems.

Our numerical simulations reveal that in general, the new neural network is more dominant than all other neural networks in the SOCCVI literature in terms of stability and convergence rates of trajectories to SOCCVI solution.

Index Terms—variational inequality; second-order cone; met- ric projector; neural network; stability; second-order sufficient condition.

I. INTRODUCTION

M

ANY problems in mathematical sciences such as en- gineering, optimization, operations research, and eco- nomics, among others, can be cast as variational inequalities (VI). For instance, complementarity problems and some fixed point problems correspond to specific instances of variational inequalities. A detailed discussion of solution methods for variational inequalities can be found in [16], [19].

In this paper, we solve the SOCCVI problem: Given a mapping F : IRⁿ→ IRⁿ and a subset C ⊆ IRⁿ given as

C = {x ∈ IRⁿ| h(x) = 0, −g(x) ∈ K} ,

where h : IRⁿ→ IR^l(l ≥ 0) and g : IRⁿ → IR^m(m ≥ 1), the SOCCVI problem is to obtain a point x ∈ C with the property that for all y ∈ C,

hF (x), y − xi ≥ 0. (1)

Here, h·, ·i is the usual inner product, and K is given by K = K^m1× · · · × K^mp, (2) where mi ≥ 1, m1+ · · · + mp = m and each K^mi is a second-order cone (SOC),

K^mi:= {(xi1, xi2, . . . , xim_i)^T ∈ IR^mi| k(xi2, . . . , xim_i)k ≤ xi1}

J Sun is supported by National Natural Science Foundation of China (Grant No.11301348).

JS Chen is supported by Ministry of Science and Technology, Taiwan.

J Sun and W Fu are with Shenyang Aerospace University, Shenyang 110136, China

JH Alcantara and JS Chen are with the Department of Mathemat- ics, National Taiwan Normal University, Taipei 11677, Taiwan (e-mail:

80640005S@ntnu.edu.tw; jschen@math.ntnu.edu.tw )

where k · k is the usual Euclidean norm and K¹is defined to be the set of nonnegative real numbers. Note that a special case of (2) is when p = n and m1 = · · · = mp = 1, which corresponds to the nonnegative orthant K = IRⁿ₊. Throughout the paper, we assume continuous differentiability of F , and twice continuous differentiability of h and g. We also denote g(x) = (g_m1(x), · · · , g_mp(x))^T and gmi = (gⁱ₀, ¯gⁱ) : IRⁿ → IR^mi for i ∈ {1, · · · , p}.

A convex second-order cone program (CSOCP), which is given by

min f (x) s.t. Ax = b

−g(x) ∈ K

(3)

is a special case of the SOCCVI (1). In (3), we assume that f : IRⁿ → IR is a twice continuously differentiable convex function, g : IRⁿ→ IR^mis differentiable, A is an l × n matrix with full row rank, and b ∈ IR^l. Indeed, by looking into the KKT conditions, the CSOCP (3) is equivalent to the SOCCVI problem (1) with F (x) = ∇f (x) and h(x) = Ax − b. This special case has wide applications in management science and engineering [1], [25], [28].

Because of various applications, there have been significant research efforts on computational approaches to variational inequalities and complementarity problems; see [5], [7], [9], [12], [16], [19], [38] and references therein. One main issue, however, is that these methods usually do not provide real- time solutions, which is necessary especially in scientific and engineering applications. Fortunately, we can obtain real-time solutions by utilizing neural networks applied to optimization.

This approach was first introduced by Hopfield and Tank [20], [37] in the field of optimization, and since then has been applied to several optimization problems; see [4], [8], [13], [14], [15], [18], [21], [22], [23], [24], [27], [39], [40], [41], [43], [44] and references therein. In this approach, the key is to set up an energy function, which is then used to formulate a system of first-order differential equations, which is a representation of an artificial neural network.

Under stability conditions, the neural network converges to a stationary solution of the differential equation, which in turn is a possible solution to the mathematical programming problem.

Neural networks have already been used to solve the CSOCP (3), which is a special case of (1) as mentioned earlier.

In [25], two kinds of neural networks for CSOCP (3) where g(x) = −x using the smoothed Fischer-Burmeister (FB) function and the projection mapping were proposed. More general neural models to efficiently solve (3) were proposed in [29], [30]. Meanwhile, there has also been a plethora

of research works making use of neural models to solve more general variational inequalities; see [22], [23], [39] and references therein. However, in the case of SOCCVI (1), only four neural networks exist in the literature. The first two of which were designed in [34]. One of them is constructed using the FB function to obtain a merit function for corresponding KKT conditions, while the other one is constructed by using a projection map to obtain a reformulation of the SOCCVI as a system of equations. In both models, the equilibrium solutions of the network are candidate solutions of (1). The other two neural networks which were used in [35] are inspired by the construction of the first neural network in [34]. Instead of the FB function, two newly discovered SOC-complementarity functions of discrete-type were used to construct the merit functions. Recently, a neural network which is supposed to solve SOCCVI (1) was proposed in [31]. However, we wish to point out that the presented model in [31] is in fact equipped to solve only the CSOCP (3).

In summary, the current literature on SOCCVI problem is very limited and the analysis of existing models that have been studied so far are based on the first-order necessary conditions. To our knowledge, there is no existing literature on second-order sufficient conditions for the SOCCVI problem.

Apart from limitations of theoretical analysis to first-order conditions, the above-mentioned neural networks considered in [34], [35] have some disadvantages such as sensitivity to initial conditions, oscillating solutions, and long convergence time.

One other major shortcomings of these neural models is their complete failure to solve some SOCCVI. Motivated by these, we present another neural network for solving the SOCCVI problem based on the smoothing metric projector. One main theoretical contribution of this paper arising from formulating this new neural network is the exploration of second-order conditions to achieve exponential stability, which has not been done in the past as mentioned above. On the other hand, from a numerical point of view, the major merit of the proposed neural network is that it addresses the inadequacies and shortcomings of the current models in [34], [35].

This paper is organized as follows: In Section 2, we present some mathematical preliminaries pertaining to the second order cone. In Section 3, we present our new neural network and provide conditions to achieve different kinds of stability.

We shall note that the stability analysis of the network is analogous to the analyses presented in our earlier works [34], [35]. However, we also present in Section 4 a rigorous analysis on how to achieve the conditions which are required to obtain a special type of stability, namely exponential stability. In particular, it is well-known in the neural network literature that nonsingularity is significant to guarantee exponential stability. Hence, we look at the Jacobian of the KKT system corresponding to (1) and provide a sufficient requirement for its nonsingularity. Finally, in Section 5, we provide numerical reports on the performance of the neural network in solving the SOCCVI.

II. PRELIMINARIES

In this section, we review important concepts associated with second-order cones (2). Most of these materials can be found in [3].

For any two vectors x = (x0, ¯x) and y = (y0, ¯y) in IR × IR^m−1, the Jordan product of x and y is denoted by x ◦ y :=

(x^Ty, y₀x + x¯ ₀y). With this Jordan product, the pair (IR ׯ IR^m−1, ◦) is a Jordan algebra with e = (1, 0, ..., 0)^T ∈ IR × IR^m−1. We shall denote x◦x by x², which is known to belong to K^mfor any x ∈ IR^m. The square root of a vector in K^mis also well-defined, since there always exists a unique point in K^m(which we denote by x^1/2or√

x) such that x = (x^1/2)^1/2. We also denote |x| := (x²)^1/2.

Any x = (x₀, ¯x) ∈ IR × IR^m−1 has the following spectral decomposition:

x = λ₁(x)c₁(x) + λ₂(x)c₂(x), (4) where λ1, λ2 are the spectral values of x with formulas

λi(x) = x0+ (−1)ⁱk¯xk (i = 1, 2) (5) while c1, c2 are the spectral vectors associated with x given by

c_i(x) =

 1

2(1, (−1)ⁱ_k¯^¯x_xk), if x 6= 0,¯

1

2(1, (−1)ⁱw), if x = 0,¯ (i = 1, 2) (6) where w is an arbitrary unit vector in IR^m−1.

Given the spectral decomposition of x as in (4), the projec- tion Π_K^m(x) of x onto K^mis

Π_Km(x) = max{0, λ₁(x)} c₁(x)+max{0, λ₂(x)} c₂(x). (7) Indeed, plugging in λi(x) and ci(x) given in (5) and (6), respectively, yields

ΠK^m(x) =







1

2(1 +_k¯^x_xk⁰ )(k¯xk , ¯x), if |x0| < k¯xk , (x0, ¯x), if k¯xk 6 x⁰,

0, if k¯xk 6 −x⁰.

The following proposition gives a formula for the directional derivative of the mapping given by (7). In what follows, we denote by intK, bdK and clK the interior, boundary, and closure of a set K ⊂ IRⁿ, respectively.

Lemma 2.1: [32, Lemma 2] ΠK^m(·) is directionally dif- ferentiable at x for any d ∈ IR^m. Moreover, the directional derivative is described by

Π⁰_Km(x; d) =























J Π_Km(x)d, d,

d − 2c1(x)^Td

− c1(x), 0,

2c2(x)^Td

+ c₂(x), ΠK^m(d),

if x ∈ IR^m\(K^m∪ −K^m), if x ∈ intK^m,

if x ∈ bdK^m\{0}, if x ∈ −intK^m, if x ∈ −bdK^m\{0}, if x = 0,

where

J Π_Km(x) = 1 2

1 _k¯^x¯_xk^T

¯ x

k¯xk I +_k¯^x_xk⁰ I −_k¯^x_xk⁰ ·_k¯^x¯¯_xk^xT2

! ,

c1(x)d^T

− := min0, c1(x)^Td ,

c2(x)d^T

+ := max0, c2(x)^Td .

For convenience in subsequent discussions, we state the definitions of the tangent cone, regular and normal cone of a closed set at a point. These concepts can be found in [33].

For a closed set K ⊆ IRⁿ and a point ¯x ∈ K, we define the following sets:

(a) the tangent (Bouligand) cone TK(¯x) := lim sup

t↓0

K − ¯x t , (b) the regular (Fr´echet) normal cone

NˆK(¯x) := { v ∈ Rⁿ| hv, y − ¯xi 6 o(ky − ¯xk), ∀y ∈ K}, (c) the limiting (in the sense of Mordukhovich) normal cone

NK(¯x) := lim sup

x→¯Kx

NˆK(x).

When K is a closed convex set, it is known that TK(¯x) = cl(K + IR¯x) and Nˆ_K(¯x) = N_K(¯x) = T_K(¯x)^◦ = { v ∈ K^◦| hv, xi 6 0}, where K^◦ denotes the polar of K.

The tangent and second-order tangent cones are explicitly known as stated in the following result.

Lemma 2.2:[2, Lemma 2.5] The tangent and second-order tangent cones of K^m at x ∈ K^m are described, respectively, by

TK^m(x) =





 IR^m, K^m,

{d = (d0, ¯d) ∈ IR × IR^m−1|h ¯d, ¯xi − x0d06 0},

if x ∈ intK^m, if x = 0, if x ∈ bdK^m\{0}.

and

T_K²m(x, d) =





 IR^m, TK^m(d),

{w = (w₀, ¯w) ∈ IR × IR^m−1|h ¯w, ¯si − w₀x₀6 d²0− k ¯dk²},

if x ∈ intT_Km(x), if x = 0,

otherwise.

We close this section by introducing some notations that will be used throughout the paper. Given a sequence {tn} ∈ R, we write tn ↓ 0 to mean that {tn} is monotone decreasing and converges to zero. The distance from a point x to a set K ⊂ IRⁿ, denoted by dist(x, K) is given by

dist(¯x, K) := inf{k¯x − ¯yk : ∀¯y ∈ K}.

By linK, we mean the linear subspace generated by K. Given x, y ∈ IRⁿ, we write x ⊥ y if and only if hx, yi = 0. For a function f : IRⁿ → IR, we denote by ∇f (x) and ∇²f (x) the gradient and Hessian of f , respectively. Finally, given a function F : IRⁿ → IR^m, we denote by J F (x) the Jacobian of F and we let ∇F (x) = J F (x)^T. To emphasize that the derivative is taken w.r.t. x, we write JxF (x) and ∇xF (x), respectively.

III. THE MODEL AND STABILITY ANALYSIS

Similar to the neural networks in [34], [35], we use the KKT conditions of the SOCCVI (1) to construct a neural network.

Recall that the variational inequality Lagrangian function is given by

L(x, µ, λ) = F (x) + ∇h(x)µ + ∇g(x)λ, (8) with µ ∈ IR^l and λ ∈ IR^m. Then the KKT system of (1) is described by







L(x, µ, λ) = 0,

hg(x), λi = 0, −g(x) ∈ K, λ ∈ K, h(x) = 0.

(9)

We formulate a neural network which can solve the system (9), which are the candidate solutions of the SOCCVI (1).

First, to achieve the complementarity requirement in system (9), we may use an SOC-complementarity function φ : IR^m× IR^m→ IR^m, i.e. a function such that φ(x, y) = 0 if and only

if x ∈ K^m, y ∈ K^m and hx, yi = 0. Two popular examples are the FB function

φ_FB(x, y) := (x²+ y²)^1/2− (x + y), and the natural residual (NR) function [17]

φ_NR(x, y) := x − Π_K^m(x − y), (10) where Π_K^m is the metric projector given by (7). Both of these functions are nonsmooth. In [34], a smoothed Fischer- Burmeister (FB) function given by

φ^ε_FB(x, y) = (x²+ y²+ ε²e)^1/2− (x + y) (11) was employed to construct a merit function for (9), which was the basis to design the neural network involving a smoothing parameter ε. We do note that φ^ε_FB is not an SOC- complementarity function.

On the other hand, “discrete” generalizations of the FB and NR function were used in [35] to design neural networks, which are given respectively by

φ^p

D−FB(x, y) =p

x²+ y²^p

− (x + y)^p, (12) and

φ^p

NR(x, y) = x^p− [(x − y)₊]^p, (13) where p > 1 is an odd integer in both cases. These dis- crete generalizations are continuously differentiable functions, which makes them suitable for neural network approaches.

In this paper, we use a smoothed natural residual function to design a neural model. We begin with a smoothing metric projector function Φ : IR+× IR^m→ IR^m given by

Φ(ε, u) := 1 2(u +p

ε²e + u²), ∀(ε, u) ∈ IR+× IR^m. (14) Observe that Φ(0, u) = ΠK^m(u). Moreover, Φ is continuously differentiable on any neighborhood of (ε, u) ∈ IR × IR^m

provided that (ε²e + u²)₀ 6=

ε²e + u²

. From [23], it is known that Φ is globally Lipschitz continuous and is strongly semismooth for all (0, u) ∈ IR × IR^m. Furthermore, applying the concept of SOC-functions in [10], [11], [6], it can be verified that the function Φ(ε, u) given in (14) can alternatively be expressed as

Φ(ε, u) = φ(ε, λ1)c1+ φ(ε, λ2)c2, (15) where φ(ε, t) := ¹₂(t+√

ε²+ t²), where λiand ciare given in (5) and (6), respectively. Hence, we can write out the function Φ as

Φ(ε, u)

=











1

2u + ¹₄ pε²+ λ²₁+pε²+ λ²₂

pε²+ λ²₁+pε²− λ²₂

¯ u k¯uk

!

, if ¯u 6= 0,

1 2

 u0+pε²+ u²₀ 0



, if ¯u = 0.

(16) For (ε²e + u²)₀6=

ε²e + u²

, we calculate the derivative of Φ w.r.t. ε as below:

∇εΦ(ε, u) = 1 2

 ∂

∂εφ(ε, λ1)c^T₁ + ∂

∂εφ(ε, λ2)c^T₂



=1 2

εc^T₁

pε²+ λ²₁+ εc^T₂ pε²+ λ²₂

!

As for the differential with respect to u, we have two cases:

(i) For u 6= 0,

∇uΦ(ε, u) = 1 2



 1 +¹₂



λ1

√

ε²+λ²₁+√^λ2

ε²+λ²₂

 Y^T

Y Z



, (17) where

Y = 1 2

"

λ2

pε²+ λ²₂ − λ1

pε²+ λ²₁

# u¯ k¯uk and

Z =

"

1 + pε²+ λ²₂−pε²+ λ²₁ λ2− λ1

# Im−1

+

"

1 2

λ1

pε²+ λ²₁ + λ2

pε²+ λ²₂

!

− pε²+ λ²₂−pε²+ λ²₁ λ2− λ1

# u¯¯u^T k¯uk²; (ii) For ¯u = 0,

∇uΦ(ε, u) = 1 2

"

1 + u0

pε²+ u²₀

# Im.

For (ε²e + u²)0 =

ε²e + u²

, Φ is nonsmooth at (ε, u) but its B-subdifferential can nevertheless be computed.

According to the above Φ(ε, u) given in (14), (15) or (16), we introduce the smoothing NR function given as

φ^ε

NR(x, y) = x − Φ(ε, x − y), (18)

which is the basis of our neural network. Now, define S : IR × IRⁿ× IR^l× IR^m→ IR × IRⁿ× IR^l× IR^mby

S(z) =



















ε L(x, µ, λ)

h(x) φ^ε_NR(−gm1(x), λm1)

... φ^ε

NR −gmp(x), λ_mp

















 ,

where z = (ε, x, µ, λ) ∈ IR × IRⁿ× IR^l× IR^m. Then, it is clear to see that solving (9) is equivalent to solving the problem

min Ψ(z) := 1

2kS(z)k². (19)

Hence, Ψ is a merit function for (9) and in turn, we consider the dynamical system given by

( dz(t)

dt = −ρ ∇Ψ(z(t)) = −ρ∇S(z(t))S(z(t)), z(t0) = z0,

(20) where ρ > 0 is a scaling factor, for solving the SOCCVI.

We refer to the above as “the smoothed NR neural network”.

The block diagram of the above neural network is presented in Figure 1. The circuit for (20) requires n + l + m + 1 integrators, n processors for F (x), m processors for g(x), mn processors for ∇g(x), l processors for h(x), ln processors for ∇h(x), (1 + m + l)n² processors for ∇xL(x, µ, λ), 2m + 2Pp

i=1m²_i processors for Φ and its derivatives, and some analog multipliers and summers.

Fig. 1: Block diagram of the proposed neural network with φ^ε

NR.

Let umi = −g_mi(x) − λ_mi. For subsequent use in the

numerical simulations, we shall note that

∇S(z) =







 1 0 0 0

0

∇xL(x, µ, λ)^T

∇h(x)^T

∇g(x)^T 0

∇h(x) 0 0

{−∇εΦ(ε, umi)}^pi=1

−∇g(x) I − diag{∇u_miΦ(ε, umi)}^p_i=1
0

diag{∇u_miΦ(ε, umi)}^p_i=1









=







 1 0 0 0

0

∇xL(x, µ, λ)^T

∇h(x)^T

∇g(x)^T 0

∇h(x) 0 0

{−∇εΦ(ε, −gmi(x) − λmi)}^p_i=1

−∇g(x) I + diag{∇g_miΦ(ε, −gmi(x) − λmi)}^p_i=1
0

−diag{∇λ_miΦ(ε, −gmi(x) − λmi)}^p_i=1







 .

It is clear that Ψ is a nonnegative function which attains the value 0 at z = (ε, x, µ, λ) if and only if (x, µ, λ) is a KKT point. Moreover, KKT points are equilibrium points of (20), and the converse holds if we have the nonsingularity of ∇S(z).

The stability analysis of the above system (20) is fairly standard and is analogous to the analysis of the smoothed FB neural network in [34]. However, we point out that our main contributions are: (i) In Section 4, we look into second- order sufficient conditions for nonsingularity; (ii) In Section 5, we demonstrate that our neural model has better numerical properties among all neural networks for SOCCVI problems.

For the sake of completeness, we present here a fundamental stability result, whose proof is similar to earlier works (for instance, [34]) and is therefore omitted.

Theorem 3.1:Isolated equilibrium points of (20) are asymp- totically stable. Moreover, we obtain exponentially stability if

∇S(z) is nonsingular.

From the above theorem, we see the importance of nonsin- gularity of the transposed Jacobian of S, namely ∇S(z). We explore sufficient conditions to guarantee this property in the next section.

IV. SECOND-ORDER SUFFICIENT CONDITION AND NONSINGULARITY THEOREM

This section is devoted to deriving the second-order suf- ficient condition for (1) and building up some conditions to achieve the nonsingularity of ∇S(0, x^∗, µ^∗, λ^∗). To this end, we write out the first-order optimality conditions for the SOCCVI problem (1). Let L(x, µ, λ) be given by (8) and let (µ, λ) = (µ, λm1, · · · , λmp) ∈ IR^l× IR^m1 × · · · × IR^mp = IR^l × IR^m. Suppose that x^∗ is a solution of (1), and the Robinson’s constraint qualification

 ∇h(x^∗)^T

−∇g(x^∗)^T



IRⁿ+ T_{0l}×K(h(x^∗), −g(x^∗)) = IR^l× IR^m holds at x^∗. The first-order optimality condition is

hF (x^∗), di ≥ 0, ∀d ∈ TC(x^∗), (21) where

TC(x^∗) =d | ∇h(x^∗)^Td = 0, −∇g(x^∗)^Td ∈ TK(−g(x^∗)) . It is known that TC(x^∗) is convex and

NC(x^∗) = ∇h(x^∗)IR^l+ {∇g(x^∗)λ | − λ ∈ NK(−g(x^∗))} , where NK(y) := NKm1(ym1) × NKm2(ym2) × · · · × NK^mp(ymp) for y = (ym1, . . . , ymp) ∈ IR^m, and

N_Kmi(y_mi) := {u_mi∈ IR^mi|humi, v − y_mii ≤ 0, ∀v ∈ K^mi

is the normal cone of Kmi at ymi. Note that (21) holds if and only if 0 ∈ F (x^∗)+N_C(x^∗) which is equivalent to: ∃ µ ∈ IR^l, λ ∈ IR^msuch that

L(x^∗, µ, λ) = 0, −λ ∈ NK(−g(x^∗))

and the set of multipliers (µ, λ) denoted by Λ(x^∗) is nonempty compact. Therefore, x^∗ satisfies the following Karush-Kuhn- Tucker condition,







L(x^∗, µ, λ) = 0, h(x^∗) = 0,

−λ ∈ N_K(−g(x^∗)).

Using the metric projector and the definition of the normal cone, the KKT condition can be expressed as

S(x, µ, λ) =





L(x, µ, λ) h(x)

−g(x) − Π_K(−g(x) − λ)



= 0, where

ΠK(−g(x) − λ) := ΠKm1(−g_m1(x) − λ_m1)^T, · · · , ΠK^mp(−g_mp(x) − λ_mp)^T
T

. It is particularly emphasized that

Π⁰_K(−g(x) − λ; d) := diag{Π⁰_Km1(−gm_i(x) − λm_i; dm_i)}^p_i=1, for d ∈ R^m.

Before presenting our main results, we recall the following concept needed in the proof.

Definition 4.1:[2] The critical cone at x^∗ is defined by C(x^∗) = {d | d ∈ T_C(x^∗), d⊥F (x^∗)} .

Theorem 4.1: Suppose that x^∗ is a feasible point of the SOCCVI (1) such that Λ(x^∗) = {(µ, λ)} is nonempty and compact. If J F (x^∗) is positive semidefinite and Robinson’s CQ holds at x^∗, then

sup

(µ,λ)∈Λ(x^∗)

hJxL(x^∗, µ, λ)d, di − δ^∗(λ | T_K²(−g(x^∗), −∇g(x^∗)^Td)) > 0, ∀d ∈ C(x^∗)\{0}

(22) is the second-order sufficient condition of (1),where

δ^∗(λ | T_K²(−g(x^∗), −∇g(x^∗)^Td)) =

 0, if λ ∈ N_K(−g(x^∗)) and hλ, −∇g(x^∗)^Tdi = 0;

+∞, otherwise.

Proof: Let x^∗ be a solution of (1). Since J F (x^∗) is positive semidefinite, we see that for some small ε > 0,

hF (x^∗), x − x^∗i > 0, ∀x ∈ Bε(x^∗) ∩ C,

where Bε(x^∗) denotes the ε-neighborhood of x^∗. Equivalently, x^∗∈ arg min{hF (x^∗), x − x^∗i | x ∈ Bε(x^∗) ∩ C} (23) Again, due to J F (x^∗) being positive semidefinite, it is clear that (23) holds if and only if

x^∗∈ arg min {hF (x^∗), x − x^∗i + hJ F (x^∗)(x − x^∗), x − x^∗i | x ∈ Bε(x^∗) ∩ C} .

(24) Therefore, we turn to deduce the second-order sufficient condition of (24). To this end, we consider the optimization problem

min hF (x^∗), x − x^∗i +¹₂hJ F (x^∗)(x − x^∗), x − x^∗i s.t. x ∈ B^ε(x^∗) ∩ C.

(25)

First, it is known that x^∗ is the stationary point of problem (25) if and only if

0 ∈ F (x^∗) + J F (x^∗)(x − x^∗) + N_Bε(x^∗_)∩C(x^∗) (26) where

N_Bε(x∗)∩C(x^∗) = N_Bε(x∗)(x^∗) + N_C(x^∗) = N_C(x^∗) (27) On the other hand, (26) and (27) imply that 0 ∈ F (x^∗) + NC(x^∗). Hence, if x^∗ is a solution of (1), we conclude that x^∗ is the stationary point of problem (25).

Now, we prove that the critical cones Cp(x^∗) and C(x^∗) of (25) and (1), respectively, are equal. Indeed,

Cp(x^∗) =





 d ∈ Rⁿ

     





∇h(x^∗)^Td

−∇g(x^∗)^Td d



∈ T_{{0}×K×Bε(x}∗)(h(x^∗), −g(x^∗), x^∗), and

hd, F (x^∗) + J F (x^∗)(x − x^∗)i = 0







Notice that

T_{{0}×K×Bε(x}^∗₎(h(x^∗), −g(x^∗), x^∗)

= T_{0}×K(h(x^∗), −g(x^∗)) × T_Bε(x∗)(x^∗)

= T_{0}×K(h(x^∗), −g(x^∗)) × Rⁿ. This yields that

C_p(x^∗) =

 d ∈ Rⁿ

   

 ∇h(x^∗)^Td

−∇g(x^∗)^Td



∈ T_{0}×K(h(x^∗), −g(x^∗)), hd, F (x^∗)i = 0



= C(x^∗).

Next, the Lagrange function of problem (25) is

L(x^∗, λ, µ, ν) = hF (x^∗), (x − x^∗)i +¹₂hJ F (x^∗)(x − x^∗), x − x^∗i + hh(x), µi + hg(x), λi + hx, νi

which gives

∇_xL(x^∗, λ, µ, ν) = F (x^∗) + J F (x^∗)(x − x^∗) + ∇h(x)µ + ν + ∇g(x)λ

∇²_xxL(x^∗, λ, µ, ν) = J F (x^∗) + Σ^l_i=1µ_i∇²h_i(x^∗) + Σ^m_i=1λ_i∇²g_i(x^∗) Here, we note that ∇²_xxL(x^∗, λ, µ, ν) = JxL(x^∗, λ, µ).

On the other hand, in light of [3, Proposition 3.269], we can check that {0} × K is second order regular at (h(x^∗), −g(x^∗)) along the direction (∇h(x^∗)^Td, −∇g(x^∗)^Td) with respect to the mapping

 ∇h(x^∗)^T

−∇g(x^∗)^T



for all d ∈ C(x^∗). Then, using the definition of the second-order regularity (see [3, Definition 3.85]) yields

y_n=

 h(x^∗)

−g(x^∗)

 + t_n

 ∇h(x^∗)^Td

−∇g(x^∗)^Td

 +¹

2t_n²r_n, ∀yn∈ {0} × K,

where tn ↓ 0, rn=

 ∇h(x^∗)^Twn

−∇g(x^∗)^Twn



+ an with an being a convergent sequence and tnwn→ 0, (n → +∞) such that

n→∞lim dist(rn, T²{0} ×K((h(x^∗), −g(x^∗)), (∇h(x^∗)^Td, −∇g(x^∗)^Td))) = 0.

According to the above result, for all Pn∈ {0} × K × Bε(x^∗), we have

P_n=



 h(x^∗)

−g(x^∗) x^∗



+t_n





∇h(x^∗)^Td

−∇g(x^∗)^Td d



+1 2t_n²

 rn

qn

 ,

where, tn ↓ 0,

 rn

qn



=





∇h(x^∗)^Twn

−∇g(x^∗)^Twn

wn



+

 an

bn



with

 an

bn



being a convergent sequence and tnwn → 0, (n → +∞). Therefore, we obtain

lim

n→∞dist(r_n, T²_{{0} ×K}((h(x^∗), −g(x^∗)), (∇h(x^∗)^Td, −∇g(x^∗)^Td))) = 0 and

n→∞lim dist

rn

qn



, T²_{{0} ×K×Bε}(x^∗)((h(x^∗), −g(x^∗), x^∗), (∇h(x^∗)^Td, −∇g(x^∗)^Td, d))



= lim

n→∞dist

rn

qn



, T²{0} ×K((h(x^∗), −g(x^∗)), (∇h(x^∗)^T, −∇g(x^∗)^Td)) × T²_Bε(x^∗)(x^∗, d)



= lim

n→∞dist(rn, T²{0} ×K((h(x^∗), −g(x^∗)), (∇h(x^∗)^Td, −∇g(x^∗)^Td)))

= 0,

and thus, {0} × K × Bε(x^∗) is second-order regular at the point (h(x^∗), −g(x^∗), x^∗) along (∇h(x^∗)^Td, −∇g(x^∗)^Td, d) with respect to the mapping





∇h(x^∗)^T

−∇g(x^∗)^T I



 for all d ∈ C(x^∗), with I as the identity map.

This together with [3, Theorem 3.86] indicates that for (25), the second-order sufficient condition is

sup

(λ,µ,ν)∈ ¯Λ(x^∗)

n∇²xxL(x^∗, λ, µ, ν) − δ^∗((µ, λ, ν), T²_{0}×K×Bε(x^∗)((h(x^∗), −g(x^∗), x^∗),

(∇h(x^∗)^Td, −∇g(x^∗)^Td, d))) o

> 0, ∀d ∈ Cp(x^∗)\{0}.

We can further simplify it as

sup

(λ,µ,ν)∈ ¯Λ(x^∗)

n∇²_xxL(x^∗, λ, µ, ν)(d, d) − δ^∗((µ, λ, ν), T²_{0}×K×Bε(x^∗)((h(x^∗), −g(x^∗), x^∗),

(∇h(x^∗)^T, −∇g(x^∗)^Td, d)))o

= sup

(λ,µ,ν)∈ ¯Λ(x^∗)

n∇²_xxL(x^∗, λ, µ, ν)(d, d) − δ^∗((µ, λ, ν), T²{0}(h(x^∗), ∇h(x^∗)^Td)

×T²K(−g(x^∗), −∇g(x^∗)^Td) × T²Bε(x^∗)(x^∗, d))o

= sup

(λ,µ,ν)∈ ¯Λ(x^∗)

n

∇²xxL(x^∗, λ, µ, ν)(d, d) − δ^∗((µ, λ, ν), {0} × T²K(−g(x^∗), −∇g(x^∗)^Td × IRⁿ)o

= sup

(µ,λ)∈Λ(x^∗)

n

JxL(x^∗, λ, µ)(d, d) − δ^∗(λ|T²K(−g(x^∗), −∇g(x^∗)^Td))o .

To sum up, the second-order sufficient condition of the SOC- CVI (1) is described by

sup

(µ,λ)∈Λ(x^∗)

hJxL(x^∗, λ, µ)d, di − δ^∗(λ|T²K(−g(x^∗), −∇g(x^∗)^Td)) > 0, ∀d ∈ C(x^∗)\{0},

as desired.

As we saw in Theorem 3.1, ∇S(0, x^∗, µ^∗, λ^∗) being non- singular is crucial to guarantee that the equilibrium point of our network becomes a solution of the SOCCVI (1) and that it is exponential stable. Now, we present some conditions to achieve the nonsingularity of ∇S(0, x^∗, µ^∗, λ^∗).

Theorem 4.2: Suppose (x^∗, µ^∗, λ^∗) is a KKT point of (1).

Then, ∇S(0, x^∗, µ^∗, λ^∗) is nonsingular if (i) Λ(x^∗) 6= ∅;

(ii) the second-order sufficient condition (22) holds;

(iii) −λ^∗∈ intN_K(−g(x^∗)) holds; and

(iv) the following constraint nondegeneracy holds:

 ∇h(x^∗)^T

−∇g(x^∗)^T



IRⁿ+linT_{0l}×K(h(x^∗), −g(x^∗)) = IR^l×IR^m.