Neural networks based on three classes of NCP-functions for solving nonlinear complementarity problems R

ContentslistsavailableatScienceDirect

Neurocomputing

journalhomepage:www.elsevier.com/locate/neucom

Neural networks based on three classes of NCP-functions for solving nonlinear complementarity problems ^R

Jan Harold Alcantara, Jein-Shan Chen

^∗

Department of Mathematics National Taiwan Normal University Taipei 11677, Taiwan

a rt i c l e i nf o

Article history:

Received 5 February 2019 Revised 15 April 2019 Accepted 29 May 2019 Available online 31 May 2019 Communicated by Dr Q Wei Keywords:

NCP-function

Natural residual function Complementarity problem Neural network Stability

a b s t ra c t

Inthispaper,weconsiderafamilyofneuralnetworksforsolvingnonlinearcomplementarityproblems (NCP). Theneural networks areconstructedfromthe merit functions basedonthree classes ofNCP- functions:thegeneralizednaturalresidualfunctionanditstwosymmetrizations.Inthispaper,wefirst characterizethe stationarypointsoftheinducedmerit functions.Growthbehaviorofthecomplemen- tarityfunctionsisalsodescribed,asthiswillplayanimportantroleindescribing thelevelsetsofthe meritfunctions.Inaddition,the stabilityofthesteepestdescent-basedneuralnetworkmodel forNCP isanalyzed.Weprovidenumericalsimulationstoillustratethetheoreticalresults,andalsocomparethe proposedneuralnetworkswithexistingneuralnetworksbasedonotherwell-knownNCP-functions.Nu- mericalresultsindicatethattheperformanceoftheneuralnetworkisbetterwhentheparameterpas- sociatedwiththeNCP-functionissmaller.TheefficiencyoftheneuralnetworksinsolvingNCPsisalso reported.

© 2019ElsevierB.V.Allrightsreserved.

1. Introductionandmotivation

Given a function F: IRⁿ→IRⁿ, the nonlinear complementarity problem(NCP)istofindapointx∈IRⁿsuchthat

x≥ 0, F

(

^x

)

^{≥ 0,}



^x,F

(

^x

) 

^{=0, (1)}

where



·,·



^{is the Euclidean inner product and} ≥ means the component-wiseorder onIRⁿ.Throughout thispaper,we assume thatFiscontinuously differentiable,andletF=

(

^F1,...,Fn

)

^{T with} F_i:IRⁿ→IRfori=1,...,n.

For decades,substantial research effortshave beendevoted in thestudyofnonlinearcomplementarityproblemsbecauseoftheir wide range of applications in many areas such as optimization, operationsresearch, engineering,andeconomics[8,9,12,48].Some sourceproblemsofNCPsincludemodels ofequilibriumproblems in the aforementioned fields and complementarity conditions in constrainedoptimizationproblems[9,12].

There are many methods in solving the NCP (1). In general, thesesolution methodsmay be categorizedinto two classes,de- pendingon whetheror not they makeuse ofthe so-calledNCP- function(seeDefinition2.1).Some techniquesthat usuallyexploit NCP-functions include merit function approach [11,19,26], nons-

R The research is supported by Ministry of Science and Technology, Taiwan.

∗ Corresponding author.

E-mail addresses: 80640 0 05s@ntnu.edu.tw (J.H. Alcantara), jschen@math.ntnu.edu.tw (J.-S. Chen).

mooth Newton method [10,45], smoothing methods [4,31], and regularization approach [17,37]. Onthe other hand, interior-point method[29,30]andproximal pointalgorithm[33]aresome well- knownapproachestosolve(1)whichdonotutilizeNCP-functions ingeneral.TheexcellentmonographofFacchineiandPang[9]pro- vides a thorough survey and discussion of solution methods for complementarityproblemsandvariationalinequalities.

Theabove numericalapproachescan efficientlysolvetheNCP;

however, itis often desirable inscientific andengineeringappli- cations to obtain a real-time solution. One promising approach thatcanprovidereal-timesolutionsistheuseofneuralnetworks, whichwerefirstintroducedinoptimizationbyHopfieldandTank inthe1980s[13,38].Neuralnetworksbasedoncircuitimplemen- tation exhibit real-time processing. Furthermore, prior researches show that neural networkscan be used efficiently in linear and nonlinear programming, variational inequalities and nonlinear complementarity problems [2,7,14,15,20,23,42–44,47,49] and as wellasinotherfields[25,28,34,36,39,40,46,50,51,55].

Motivated by the preceding discussion, we construct a new family ofneural networks basedon recently discovered discrete- type NCP-functions tosolve NCPs. Neuralnetworksbased on the Fischer-Burmeister(FB) function[23] andthegeneralizedFischer- Burmeister function [2] have already been studied. The latter NCP-functions,which havebeenextensively usedin thedifferent solutionmethods,arestronglysemismoothfunctions,whichoften provideefficientperformance[9].Inthispaper,weexploretheuse ofsmoothNCP-functionsasbuildingblocksoftheproposedneural https://doi.org/10.1016/j.neucom.2019.05.078

0925-2312/© 2019 Elsevier B.V. All rights reserved.

networks. Moreover, the NCP-functions we consider herein have piecewise-definedformulas,asopposedtotheFBandgeneralized FB functionswhich havesimple formulations.Inturn, thesubse- quent analysis is more complicated. Nevertheless, we show that theproposedneuralnetworksmayofferpromisingresultstoo.The analysis andnumerical reports in thispaper, on the other hand, pavethewayfortheuseofpiecewise-definedNCP-functions.

This paper is organized as follows: In Section 2, we revisit equivalentreformulationsoftheNCP(1)usingNCP-functions.We also elaborate on the purpose and limitations of the paper. In Section 3, we review some mathematical preliminaries related to nonlinear mappings andstabilityanalysis.We alsosummarize some important properties of the three classes of NCP-functions weusedinconstructingtheneuralnetworks.InSection 4,wede- scribethegeneralpropertiesoftheneuralnetworks,whichinclude thecharacterizationofstationarypointsoftheinducedmeritfunc- tions. In Section 5,we lookat the growth behavior ofthe three classes of NCP-functions considered. This result will be used to provetheboundednessofthelevelsetsoftheinducedmeritfunc- tions. We also prove some stabilityproperties of theneural net- works.InSection6,wepresenttheresultsofournumericalsimu- lations.Conclusionsandsomerecommendationsforfuturestudies arediscussedinSection7.

Throughoutthe paper,IRⁿ denotes the spaceofn-dimensional realcolumnvectors,IR^{m× n}denotesthespaceofm× nrealmatri- ces, andA^T denotes the transposeofA∈IR^{m× n}.For anydifferen- tiable functionf: IRⁿ→IR,

∇

^{f(x) meansthe gradientoff atx.For}

any differentiable mapping F=

(

^F1,...,Fm

)

^T:IRn→IR^m,

∇

^F

(

^x

)

= [

∇

^F1

(

^x

)

· · ·

∇

^Fm

(

^x

)

^]∈IR^n×m denotesthetransposedJacobian ofF at x. We assume that p is an odd integer greater than 1,unless otherwisespecified.

2. Overviewandcontributionsofthepaper

In this section, we give an overviewof this research. We be- ginby lookingatequivalentreformulationsofthenonlinearcom- plementarityproblem(1)usingNCP-functions,whichisdefinedas follows.

Definition2.1. Afunction

φ

^:IR× IR→IRiscalledanNCP-function ifitsatisfies

φ (

a,b

)

=0 ⇐⇒ a≥ 0, b≥ 0, ab=0.

Thewell-knownnatural-residualfunctiongivenby

φ

NR

(

a,b

)

=a−

(

^a− b

)

+=min

{

^a,b

}

is an example ofan NCP-function, whichis widely used in solv- ingNCP.Recently, in[3],thediscrete-typegeneralizationof

φ

NR is proposedanddescribedby

φ

NR^p

(

a,b

)

=a^p−

(

^a− b

)

^p₊ ^wherep>1isodd integer. (2) It is shown in [3] that

φ

_NR^{p is twice} continuously differentiable.

However, its surfaceis not symmetric,which mayresult todiffi- culties indesigning andanalyzingsolutionmethods [16].To con- querthis, twosymmetrizationsofthe

φ

_NR^{p are presentedin[1].A}

naturalsymmetrizationof

φ

_NR^{p isgivenby}

φ

S^p−NR

(

a,b

)

=



_ap−

(

^a− b

)

^{p if a}>b, a^p=b^p if a=b, b^p−

(

^b− a

)

^{p if a}<b.

(3)

The above NCP-function is symmetric, but is only differentiable on

{ (

^a,b

) |

^a=bora=b=0

}

^{.Itwashowevershownin[16]that}

φ

_S^p_−NR ^issemismooth andis directionally differentiable.The second symmetrizationof

φ

_NR^{p isdescribedby}

ψ

S^p−NR

(

^a,b

)

=



_apb^p−

(

^a− b

)

^{pbp if a}>b, a^pb^p=a^2p if a=b,

a^pb^p−

(

^{b− a}

)

^{pap if a<b, (4)}

whichpossessesbothdifferentiabilityandsymmetry.Thefunctions

φ

_NR^p ,

φ

_S^p_−NR ^and

ψ

_S^p_−NR ^{are three classes of the four} discrete-type familiesofNCP-functions whichare recentlydiscovered, together with the discrete-type generalization of the Fischer-Burmeister functiongivenby

φ

D^p−FB

(

a,b

)

=



x²+y²



p

−

(

^x+y

)

^p. (5)

Acomprehensivediscussionoftheirpropertiesispresentedin[16]. ToseehowanNCP-function

φ

^{canbeusefulinsolvingNCP(1),}

wedefine



^:IRn→IRⁿby

(

^x

)

=

⎛

⎝ φ (

^x1,F1

(

^x

))

..

φ (

^xn,.Fn

(

^x

))

⎞

⎠

. (6)

Itiseasytoseethatx^∗solvesNCP(1)ifandonlyif

(

^x∗

)

=0(see alsoProposition 4.1 (a)).Thus, the NCPis equivalentto the non- linearsystem ofequations

(

^x

)

=0.Meanwhile, if

φ

^{isan NCP-}

function,then

ψ

^:IR× IR→IR₊givenby

ψ (

a,b

)

:= 1

2

| φ (

a,b

) |

^{2 (7)}

isalsoanNCP-function.Accordingly,ifwedefine



^:IRn→IR₊by

(

^x

)

= n

i=1

ψ (

^xi,Fi

(

^x

))

=1

2

 (

^x

) 

^{2, (8)}

then the NCP can be reformulated as a minimization problem min_x∈IRn

(

^x

)

. Hence,



^{given by (8) is a merit function for the} NCP, that is, its global minimizer coincides with the solution of theNCP. Consequently, itis onlynatural toconsider thesteepest descent-basedneuralnetwork

dx

(

^t

)

dt =−

ρ∇ (

^x

(

^t

))

, x

(

^t0

)

=x⁰, (9) where

ρ

>0 is a time-scaling factor. The above neural network (9) is also motivated by the onesconsidered in [23] andin [2], where the NCP functions used are the Fischer-Burmeister (FB) functiongivenby

φ

^FB

(

a,b

)

=



a²+b²−

(

^a+b

)

, (10)

andthegeneralizedFischer–Burmeisterfunctionsgivenby

φ

FB^p

(

a,b

)

=

 (

a,b

) 

p−

(

^a+b

)

^{where p}∈

(

¹,+∞

)

, (11) respectively. We aim to compare the neural networks based on thegeneralizednatural-residualfunctions(2),(3)and(4)withthe well-studiednetworksbasedontheFBfunctions(10)and(11).

Oneofthe contributionsofthis paperliesonestablishing the theoreticalpropertiesofthegeneralizednaturalresidualfunctions.

Theseare fundamentalindesigning NCP-basedsolution methods, andinthispaper,weusetheneuralnetworkapproach.Basicprop- erties of these functions are already presented in [16]. The pur- poseof thispaperis to elaborate some more propertiesandap- plications of the newly discovered discrete-type classes of NCP- functions given by (2), (3) and (4). Specifically, we look at the propertiesoftheirinducedmeritfunctionsgivenby(8).First,itis importantforustodeterminethecorrespondencebetweentheso- lutionsofNCP(1)andthestationarypointsof



^{.Fromtheabove} discussion(also see Proposition 4.1(d)), we alreadyknowthat an NCP solution is a stationary point. On the other hand, we also wantto determinewhich stationarypointsof



^{are solutions to} the NCP. Forcertain NCP functionssuch as the Mangasarianand

Solodovfunction[19],FBfunction[11]andgeneralizedFBfunction [5],astationarypointofthemeritfunctionwasshowntobeaso- lutionto theNCPwhenFismonotone ora P₀-function.Itshould bepointedoutthattheseNCP-functionspossessthefollowingnice properties:

(P1)

∇

^a

ψ

^(a,b)·

∇

b

ψ

^(a,b)≥ 0forall(a,b)∈IR²;and

(P2)For all (a, b)∈IR²,

∇

^a

ψ (

^a,b

)

=0⇐⇒

∇

b

ψ (

^a,b

)

=0⇐⇒

φ (

^a,b

)

=0.

However, thesepropertiesarenotpossessedby

φ

_NR^p ,

φ

_S^p_−NR ^and

ψ

_S^p_−NR,whichleadstosomedifficultiesinthesubsequentanalysis.

Hence, we seekforother conditions which will guaranteethat a stationarypointisan NCPsolution.Furthermore,we alsowantto lookatthegrowthbehaviorofthefunctions(2),(3)and(4).This willplay akeyroleincharacterizingthelevelsets oftheinduced merit functions. It must be noted that since the NCP functions

φ

_S^p_−NR^and

ψ

_S^p_−NR ^arepiecewise-definedfunctions,thentheanalyses oftheirgrowthbehaviorandthepropertiesoftheirinducedmerit functionsaremoredifficult,ascomparedwiththecommonlyused FBfunctions(10)and(11)whichhavesimpleformulations.

Anotherpurpose ofthispaperisto discussthestabilityprop- ertiesoftheneuralnetworksbased on

φ

_NR^p,

φ

_S^p_−NR ^and

ψ

_S^p_−NR^.We

furtherlook intodifferent examples to seethe influence ofpon theconvergenceof trajectoriesof theneural network tothe NCP solution.Finally,we comparethenumericalperformance ofthese threetypesofneural networkswithtwo well-studiedneural net- worksbasedonthe FBfunction [23] andgeneralizedFBfunction [2].

We recallthata solutionx^∗issaidto bedegenerateif

{

ⁱ

|

^x∗_i= F_i

(

^x∗

)

=0

}

^{isnotempty.Notethatifx∗isdegenerateand}

φ

^isdif-

ferentiableatx^∗,then

∇ 

^{(x∗)issingular.}Consequently,oneshould notexpectalocallyfastconvergenceofnumericalmethodsbased onsmooth NCP-functionsif thecomputed solution isdegenerate [9,18].Because ofthedifferentiability of

φ

_NR^p,

φ

_S^p_−NR ^and

ψ

_S^p_−NR ^on

thefeasibleregionoftheNCPproblem,itisalsoexpectedthatthe convergenceofthetrajectoriesoftheneural network(9)toade- generatesolutioncouldbeslow.Hence,inthispaper,wewillgive particularattentiontonondegenerateNCPs.

3. Preliminaries

In this section, we review some special nonlinear mappings, some properties of

φ

_NR^p,

φ

_S^p_−NR ^and

ψ

_S^p_−NR, as well as some tools

fromstabilitytheory indynamical systemsthatwill becrucial in ouranalysis.Webeginwithrecallingconceptsrelatedtononlinear mappings.

Definition3.1. LetF=

(

^F1,...,Fn

)

^T:IRn→IRⁿ.Then, themapping Fissaidtobe

(a) monotoneif



^x− y,F

(

^x

)

− F

(

^y

) 

≥ 0forallx,y∈IRⁿ. (b)strictly monotone if



^x− y,F

(

^x

)

− F

(

^y

) 

>0forall x,y∈IRⁿ

andx=y.

(c)strongly monotone with modulus

μ

>0 if



^x− y,F

(

^x

)

− F

(

^y

) 

≥

μ

^x− y



^2forallx,y∈IRⁿ.

(d) a P₀-function if max

1≤i≤n xi=yi

(

^xi− yi

)(

^Fi

(

^x

)

− Fi

(

^y

))

≥ 0 for all x, y∈IRⁿandx=y.

(e) aP-functionifmax

1≤i≤n

(

^xi− yi

)(

^Fi

(

^x

)

− Fi

(

^y

))

>0forallx,y∈IRⁿ andx=y.

(f) a uniform P-function with modulus

κ

>0 if max

1≤i≤n

(

^xi− y_i

)(

^Fi

(

^x

)

− Fi

(

^y

))

≥

κ

^x− y



²,forallx,y∈IRⁿ.

FromDefinition3.1,thefollowingone-sidedimplicationscanbe obtained:

Fisstronglymonotone⇒FisauniformP-function⇒Fisa P₀-function.

ItisknownthatFismonotone(resp.strictly monotone)ifand onlyif

∇

^{F(x)ispositive}semidefinite(resp.positivedefinite)forall x∈IRⁿ.Inaddition,F isaP₀-functionifandonlyif

∇

^{F(x) isaP}0- matrixforallx∈IRⁿ;that is,itsprincipalminorsarenonnegative.

Further,if

∇

^{F(x)isaP-matrix(thatis,itsprincipalminorsarepos-}

itive) forall x∈IRⁿ,thenFisa P-function.However, wepoint out that aP-functiondoesnot necessarilyhavea Jacobianwhich isa P-matrix.

The following characterization of P-matrices and P₀-matrices willbeusefulinouranalysis.

Lemma3.1. A matrixM∈IR^{n× n} isa P-matrix (resp.a P₀-matrix) if andonlyifwheneverx_i(Mx)_i≤ 0(resp.x_i(Mx)_i<0)foralli,thenx= 0.

Proof. Pleasesee[6]. 

Thefollowingtwolemmassummarizesome propertiesof

φ

_NR^p,

φ

_S^p_−NR ^and

ψ

_S^p_−NR ^{thatwillbeusefulinoursubsequentanalysis.}

Lemma3.2. Letp>1beanoddinteger.Then,thefollowinghold.

(a) Thefunction

φ

_NR^{p istwice}continuouslydifferentiable.Itsgradi- entisgivenby

∇φ

NR^p

(

a,b

)

=p



a^p−1−

(

^a− b

)

^p−2

(

^a− b

)

+

(

^a− b

)

^p−2

(

^a− b

)

+



.

(b) Thefunction

φ

_S^p_−NR ^istwice continuously differentiableon the set



^:={(a,b)|a=b}.Itsgradientisgivenby

∇φ

S^p−NR

(

a,b

)

=

p[a^p−1−

(

^a− b

)

^p−1,

(

^a− b

)

^{p−1]T if a}>b, p[

(

^b− a

)

^p−1,b^p−1−

(

^b− a

)

^{p−1]T if a}<b. Further,

φ

_S^p_−NR ^is differentiable at (0,0) with

∇φ

_S^p_−NR

(

⁰,0

)

= [0,0]^T.

(c) Thefunction

ψ

_S^p_−NR ^istwicecontinuouslydifferentiable.Itsgra- dientisgivenby

∇ψ

S^p−NR

(

a,b

)

=

⎧ ⎨

⎩

p[a^p−1b^p−

(

^a− b

)

^p−1bp, a^pb^p−1−

(

^a− b

)

^pbp−1+

(

^a− b

)

^{p−1bp]T if a}> b, p[a^p−1b^p,a^pb^p−1]^T= pa^2p−1[1,1]^T if a= b, p[a^p−1b^p−

(

^b− a

)

^pap−1+

(

^b− a

)

^p−1ap, a^pb^p−1−

(

^b− a

)

^{p−1ap]T if a}< b.

Proof. Pleasesee[3,Proposition2.2],[1,Propositions2.2and3.2], and[16,Proposition4.3]. 

Lemma3.3. Letp>1 bea positiveoddinteger. Then,thefollowing hold.

(a) If

φ

∈

{ φ

_NR^p,

φ

_S^p_−NR

}

, then

φ

^{(a, b)}>0 ⇐⇒ a>0,b>0. On the otherhand,

ψ

_S^p_−NR

(

^a,b

)

≥ 0onIR².

(b)

∇ 

^a_>

φ

_0NR^p

(

^a,_onb

) {

·

( ∇

^a,b

φ

b

)

_NR^p

| (

^aa,>b

)

b>0ora>b>2a

}

,

=0 on

{ (

^a,b

) |

^a≤ bora>b=2aora>b=0

}

,

<0 otherwise,

∇

^a

φ

_S^p_−NR

(

^a,b

)

·

∇

b

φ

_S^p_−NR

(

^a,b

)

>0 on

{ (

^a,b

) |

^a>b>0

}



{ (

^a,b

) |

^b>a>0

}

,and

∇

^a

ψ

_S^p_−NR

(

^a,b

)

·

∇

b

ψ

_S^p_−NR

(

^a,b

)

>0onthefirstquadrantIR²₊₊.

(c)If

φ

∈

{ φ

_NR^p,

φ

_S^p_−NR

}

, then

∇

^a

φ (

^a,b

)

·

∇

b

φ (

^a,b

)

=0 provided that

φ (

^a,b

)

=0. On the other hand,

ψ

_S^p_−NR

(

^a,b

)

=0⇐⇒

∇ψ

_S^p_−NR

(

^a,b

)

=0. In particular, we have

∇

^a

ψ

_S^p_−NR

(

^a,b

)

·

∇

b

ψ

_S^p_−NR

(

^a,b

)

=0providedthat

ψ

_S^p_−NR

(

^a,b

)

=0. Proof. Pleasesee[16,Propositions3.4,4.5,and5.4]. 

Next, we recall some materials about first order differential equations(ODE):

x˙

(

^t

)

=H

(

^x

(

^t

))

, x

(

^t0

)

=x⁰∈IRⁿ (12) whereH:IRⁿ→IRⁿisamapping.Wealsointroducethreekindsof stabilitythatwe willconsiderlater.Thesematerialscan befound inODEtextbooks;see[27].

Definition 3.2. A point x^∗=x

(

^t∗

)

^{is called an} equilibrium point or a steady state of the dynamic system (12) if H

(

^x∗

)

=0. If there is a neighborhood



^∗⊆IRⁿ of x^∗ such that H

(

^x∗

)

=0 and H(x)=0

∀

^x∈



^{∗\{x∗},thenx∗iscalledanisolated}equilibriumpoint.

Lemma 3.4. Assume that H: IRⁿ→IRⁿ is a continuous mapping.

Then, foranyt₀≥ 0andx₀∈IRⁿ,thereexistsa localsolutionx(t)for (12)with t∈[t₀,

τ

^{) forsome}

τ

>t₀.If,in addition,His locally Lip- schitzcontinuous atx₀,thenthesolutionis unique;ifHis Lipschitz continuousinIRⁿ,then

τ

^{canbeextendedto}∞.

Definition3.3. (StabilityinthesenseofLyapunov)Letx(t)beaso- lutionfor(12).Anisolatedequilibriumpointx^∗isLyapunovstable if foranyx₀=x

(

^t0

)

^andany

ε

>0, there exists a

δ

>0 such that



^x

(

^t

)

− x^∗



<

ε

^forallt≥ t0 and



^x

(

^t0

)

− x^∗



<

δ

^.

Definition3.4. (Asymptoticstability)Anisolatedequilibriumpoint x^∗ is said tobe asymptotically stableifin additionto beingLya- punov stable, it has the property that x(t)→x^∗ as t→∞ for all



^x

(

^t0

)

− x^∗



<

δ

^.

Definition 3.5. (Lyapunovfunction)Let



⊆IRⁿ be an openneigh- borhoodof ¯x.Acontinuously differentiablefunctionW:IRⁿ→IRis said to be a Lyapunovfunction at thestate ¯xover the set



^for Eq.(12)if



W

(

^¯x

)

=0, W

(

^x

)

>0,

∀

^x∈

 \{

^¯x

}

^.

dW

(

^x

(

^t

))

dt =

∇

^W

(

^x

(

^t

))

^TH

(

^x

(

^t

))

≤ 0,

∀

^x∈



. Lemma3.5.

(a) Anisolatedequilibriumpointx^∗isLyapunovstableifthereex- istsaLyapunovfunctionoversomeneighborhood



^∗ofx∗. (b) Anisolatedequilibriumpointx^∗isasymptoticallystableifthere

isa Lyapunovfunctionover someneighborhood



^{∗ofx∗such} that dW

(

^x

(

^t

))

dt <0 for all x∈



^∗\{x∗}.

Definition 3.6. (Exponential stability) An isolated equilibrium pointx^∗isexponentiallystableifthereexistsa

δ

>0suchthatar- bitrary point x(t)of (12)withthe initial conditionx

(

^t0

)

=x₀ and



^x

(

^t0

)

− x^∗



<

δ

^iswell-definedon[0,+∞

)

^{andsatisfies}



^x

(

^t

)

^{− x∗}



^{2≤ ce−ωt}



^x

(

^t0

)

^{− x∗}

 ∀

^{t≥ t0,}

where c>0 and

ω

>0 are constants independent of the initial point.

Thefollowingresultwillalsobehelpfulinourstabilityanalysis.

Lemma3.6. LetFbelocallyLipschitzian.IfallV∈

∂

^{F(x)arenonsingu-}

lar,thenthereisaneighborhoodN(x)ofxandaconstantCsuchthat foranyy∈N(x)andanyV∈

∂

^F(y),Visnonsingularand



^V−1



≤ C Proof. Pleasesee[32,Propositions3.1]. 

4. Neuralnetworkmodel

In thissection, we describe the properties ofthe neural net- work(9)basedonthefunctions

φ

_NR^p ,

φ

_S^p_−NR ^and

ψ

_S^p_−NR^.Beforethis,

wesummarizefirstsome importantpropertiesof



^asdefinedin (8)forgeneralNCP-functions.Proposition4.1(a)isinfactLemma 2.2in[19].Ontheotherhand,Proposition4.1(b)and(e)aretrue forallgradientsystems(9).

Proposition4.1. Let



^:IRn→IR₊bedefinedasin(8),with

φ

^being

anyNCP-function,andlet

ψ

^{beas in(7).SupposethatFiscontinu-}

ouslydifferentiable.Then,

(a)



^(x)≥ 0 for all x∈IRⁿ. If the NCP (1) has a solution, x is a globalminimizerof



^{(x)ifandonlyifxsolvestheNCP.}

(b)



^(x(t))isanonincreasingfunctionoft,wherex(t)isasolution of(9).

(c) Letx∈IRⁿ,andsupposethat

φ

^isdifferentiableat(x_i,F_i(x))for eachi=1,...,n.Then

∇ (

^x

)

=

∇

^a

ψ (

^x,F

(

^x

))

+

∇

^F

(

^x

) ∇

b

ψ (

^x,F

(

^x

))

⁽¹³⁾

where

∇

^a

ψ (

^x,F

(

^x

))

^:=[

∇

^a

ψ (

^x1,F₁

(

^x

))

,. . .,

∇

^a

ψ (

^xn,Fn

(

^x

))

^]T,

∇

b

ψ (

x,F

(

^x

))

:=[

∇

b

ψ (

^x1,F1

(

^x

))

,. . .,

∇

b

ψ (

^xn,Fn

(

^x

))

^]T. (d) Letxbe asolution totheNCPsuchthat

φ

^isdifferentiableat

(x_i,F_i(x))foreachi=1,...,n.Then,xisastationarypointof



^.

(e) Everyaccumulationpointofa solution x(t) ofneuralnetwork (9)isanequilibriumpoint.

Proof. (a)Itisclearthat



≥ 0.Noticethat

(

^x

)

=0ifandonlyif

(

^x

)

=0,whichoccursifandonlyif

φ (

^xi,F_i

(

^x

))

=0foralli.Since

φ

^isan NCP-function, thisis equivalentto having x_i≥ 0, F_i(x)≥ 0 and x_iF_i

(

^x

)

=0. Thus,

(

^x

)

=0 if and only if x≥ 0, F(x)≥ 0 and



^x,F

(

^x

) 

=0.Thisprovespart(a).

(b)Thedesiredresultfollowsfrom d

(

^x

(

^t

))

dt =

∇ (

^x

(

^t

))

^Tdx_dt =

∇ (

^x

(

^t

))

^T

(

−

ρ∇ (

^x

(

^t

)))

=−

ρ∇ (

^x

(

^t

)) 

²≤ 0 forallsolutionsx(t).

(c)Theformulaisclearfromchainrule.

(d)First, note that fromEq. (7),we have

∇ψ (

^a,b

)

=

φ (

^a,b

)

·

∇φ (

^a,b

)

^{.Thus,ifxisasolutiontotheNCP,itgives}

∇ψ (

^xi,F_i

(

^x

))

= 0forall i=1,...,n.Then, itfollowsfromformula(13)in part(c) that

∇ (

^x

)

=0.Thatis,xisastationarypointof



^.

(e)Pleaseseepage232of[41]. 

We adopt the neural network (9) with

(

^x

)

= ¹2

 (

^x

) 

², where



^{is given by (6) with}

φ

∈

{ φ

_NR^p,

φ

_S^p_−NR,

ψ

_S^p_−NR

}

^{. The func-}

tion



corresponding to

φ

_NR^p,

φ

_S^p_−NR ^and

ψ

_S^p_−NR ^{is denoted, re-}

spectively,by



_NR^p ,



_S1^p_−NR ^and



^p_S2−NR^{. Their}corresponding merit functions will be denoted by



_NR^p,



_S1^p_−NR ^and



_S2^p_−NR, respec- tively. We note that by formula (13) and the differentiability of



∈

{ 

_NR^p,



_S1^p_−NR,



_S2^p_−NR

}

^(seeProposition4.2),theneuralnetwork (9)canbeimplementedonhardwareasinFig.1.

Wefirstestablishtheexistenceanduniquenessofthesolutions ofneuralnetwork(9).

Proposition 4.2. Let p>1 be an odd integer. Then, the following hold.

(a)



_NR^{p and}



_S2^p_−NR^arebothcontinuouslydifferentiableonIRⁿ. (b)



_S1^p_−NR ^iscontinuouslydifferentiableontheopenset



=

{

^x∈

IRⁿ

|

^xi=F_i

(

^x

)

,

∀

ⁱ=1,2,...,n

}

^.