Multistability for Delayed Neural Networks
via Sequential Contracting
Chang-Yuan Cheng, Kuang-Hui Lin, Chih-Wen Shih, and Jui-Pin Tseng
Abstract— In this paper, we explore a variety of new
multistability scenarios in the general delayed neural network system. Geometric structure embedded in equations is exploited and incorporated into the analysis to elucidate the underlying dynamics. Criteria derived from different geometric configura-tions lead to disparate numbers of equilibria. A new approach named sequential contracting is applied to conclude the global convergence to multiple equilibrium points of the system. The formulation accommodates both smooth sigmoidal and piecewise-linear activation functions. Several numerical examples illustrate the present analytic theory.
Index Terms— Complete stability, delay equations, multistability, neural network.
I. INTRODUCTION
I
N RECENT decades, various neural network models have been proposed and employed in diverse applied sciences and engineering successfully. Most of the neural networks developed are based on Hopfield model [12] or Cohen–Grossberg model [8]. Delays have been considered in the neural network modeling, as time lags occur in transmitting signal among real neurons and artificial neurons. Hopfield-type neural network with delays was introduced in [25]. Later, delay has been considered and extensively studied in neural networks [1], [2], [13], [20], [29], [31], [36]. Concerning the mathematical modeling, delays can modify the collective dynamics for neural networks. Thus, in neural network models, it is appealing to investigate how the collective dynamics are determined by the connection strength, nonlinear coupling functions, and transmission delays. Although the accomplish-ments in those investigations have advanced the theories on these delayed network systems, effective approaches combined with powerful mathematical techniques are still in demand in tackling the unsolved problems, for example, to elucidate the variations of collective dynamics of the network system with respect to the size of the network, connection strength, and delay magnitude.Manuscript received August 22, 2014; accepted February 9, 2015. Date of publication March 3, 2015; date of current version November 16, 2015. This work was supported by the National Science Council of Taiwan under Grants NSC 102-2115-M-153-002 and NSC 103-2115-M-009-002-MY2.
C.-Y. Cheng is with the Department of Applied Mathematics, National Pingtung University, Pingtung 90003, Taiwan (e-mail: [email protected]).
K.-H. Lin and C.-W. Shih are with the Department of Applied Mathematics, National Chiao Tung University, Hsinchu 30010, Taiwan (e-mail: [email protected]; [email protected]).
J.-P. Tseng is with the Department of Mathematical Sciences, National Chengchi University, Taipei 11605, Taiwan (e-mail: [email protected]).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TNNLS.2015.2404801
Multistability is a notion to describe the coexistence of multiple stable equilibria or cycles. Such dynamics is essential in several applications of neural networks, including pattern recognition and associative memory storage [6], [9], [11], [12]. Hopfield-type neural networks, with or without consideration of transmission delays, have been the primary models that attract a great deal of research interests on multistability.
In this paper, we consider the general delayed neural network d xi(t) dt = −μixi(t) + n j=1 [αi jgj(xj(t)) + βi jgj(xj(t − τi j))] + Ii (1) where i = 1, . . . , n, μi > 0, αi j, βi j are connection weights from neuron j to neuron i , gj(·) are activation functions, 0 ≤ τi j are time lags that are bounded above by τM, and Ii stand for independent bias current sources. A typical class of activation functions is class A : ⎧ ⎨ ⎩ gi ∈ C2(R), gi(ξ) > 0, for all ξ ∈ R (ξ − σi)g
i(ξ) < 0, for all ξ ∈ R, ξ = σi limξ→+∞gi(ξ) = vi, limξ→−∞gi(ξ) = ui wherevi, ui, andσi are constants with ui < vi, i = 1, . . . , n. Class A contains the general bounded sigmoidal functions; a typical example is gi(ξ) = tanh ξ. The present theory also applies to (1) with piecewise-linear activation functions, such as the three-segment standard output function in cellular neural networks, gi(ξ) = [|ξ + 1| − |ξ − 1|]/2. Let ρi := max{|ui|, |vi|}, i = 1, . . . , n.
System (1) reduces to the classical and delayed Hopfield neural networks in [12] and [25], as βi j = 0 and αi j = 0 for all i, j, respectively. It also represents the cellular neural networks without delays [6] and with delays [29].
Multistability for (1) has been studied in several papers. The existence of 3n equilibria for (1) with activation functions in class A was established in [4] and [5]. Therein, it was also shown that the 2n equilibria out of those 3n equilibria are stable, and each of them is located in a positively invariant region. Later, different criteria for stability were obtained in [14] using the Lyapunov functional and matrix inequality techniques. Several works are strongly restricted to the class of piecewise-linear activation functions. For example, for (1) with standard piecewise-linear output function, it was shown in [38] that the system cannot have more than 3n equilibria, and can have 2n locally exponentially stable equilibria. Hopfield neural network with nondecreasing piecewise-linear activation
2162-237X © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
functions with 2r corner points (2r + 1 saturation regions), without time delays, was investigated in [34]. It was asserted that the n-neuron system can have and only have (2r + 1)n equilibria under some conditions, of which(r +1)nare locally exponentially stable and the others are unstable. In [40], the existence of (2k + 2m − 1)n equilibria was concluded for (1) with piecewise-linear activation functions with k+m saturation regions. Other related works include [19], [21], and [26]–[28]. Convergence of dynamics (also called complete stability), which means that every solution of the system converges to one of the equilibrium points, is a key ingredient in multistability. Quasi-convergence of dynamics for (1) was established in [5]. It indicates that almost every orbit converges to one of the 3n equilibria, and 2n among these 3n equilibria are stable, under a delay-dependent criterion. Therein, (1) is shown to be a strongly order-preserving under a special order on the space of continuous functions C([−τM, 0]). System (1) is said to be cooperative (competitive) if αi j, βi j ≥ 0 (≤ 0) for all j = i. Chua and Roska [7] have demonstrated that if the interconnection matrix is irreducible, and the neuron activations are modeled by sigmoidal functions (C1, bounded and strictly increasing), then the solution flow generated by a cooperative cellular neural network without delay is eventu-ally strongly monotone. According to the standard theory of cooperative dynamical systems, the flow enjoys the so-called limit set dichotomy, and generically, the solution converges to the set of equilibria. Marco et al. extended the limit set dichotomy to (1) with piecewise-linear activation functions, nonsymmetric cooperative interconnection matrix, and without delays [22] and with delays [23]. There were other studies on complete stability that consider only piecewise-linear activa-tion funcactiva-tions, such as [35] and [39]. Some investigaactiva-tions on convergence of neural networks with discontinuous activation functions can be found in [16] and [24].
Multistability has also been studied for Cohen–Grossberg neural networks with a class of piecewise-smooth activation functions with two saturated segments in [3]. It was shown that under some conditions, the n-neuron networks can have 2nlocally exponentially stable equilibrium points, each located in a saturated region.
The purpose of this paper is two folds. The first is to explore more multistability scenarios in the delayed neural networks (1). The second is to introduce new analytic method-ologies into the study of multistability in delayed systems. For locating an equilibrium, we develop an approach that combines the Brouwer’s fixed-point theorem or contracting mapping theorem with the geometric configurations induced from the structure of the equations. With such an approach, we shall exploit a variety of multiple equilibria for (1). For global convergence of dynamics (complete stability), we introduce a new approach named sequential contracting. We start by constructing a preliminary upper and lower dynam-ics for each component of the system. The upper and lower dynamics are designed to have their own solutions contracted to some compact intervals. We then construct finer upper and lower dynamics successively, so that the original dynamics are attracted to more concentrated regions. A criterion for contraction is then formulated so that these nested intervals
collapse into points, as the iterative constructions of upper and lower dynamics carry on. Under different formulations of lower and upper dynamics, delay-dependent criteria and delay-independent criteria for multistability of (1) can be derived, respectively. This approach can also lead to a network-scale-dependent criterion for asymptotic behaviors and synchronization in network systems [31], [32].
In previous works, as mentioned above, mathematical studies on multistability in neural networks centered around analyzing the existence of multiple equilibria using standard method or utilizing the piecewise-linear structure of the activation functions. Stability of the equilibrium and local dynamics were studied by the linearization theory or Lyapunov functional and matrix inequality techniques. On the other hand, monotone dynamics theory, which requires the interconnection matrix to be cooperative, has been adopted to conclude the global dynamics. To explore further dynamical scenarios for (1), which are embedded in the equations, new ideas are required. The sequential contracting has been applied to establish global convergence of dynamics for (1) in [30]. The dynamical scenario concluded therein is that every orbit converges to one of the 3n equilibria. As only 2n out of those 3n equilibria are stable, it was further concluded that almost every orbit converges to one of the 2nstable equilibria. In this paper, we improve the methodology to exploit further fruitful dynamics in (1). One of the main results in this paper is the existence of 3k equilibria, among which 2k equilibria are attracting, for any k≤ n, in the n-neuron network (1).
Almost all the existing results on the number of multiple equilibria are in terms of n-power of the number of saturated (or near saturated) regions in a n-neuron system, as mentioned above. Herein, the numbers of equilibria we derive are not in power of n. These new multistability scenarios demonstrate the strength of the present methodology, as they are inaccessible by other treatments. Our approach applies to both smooth sigmoidal and nondecreasing piecewise-linear activation func-tions. Thus, the formulation also covers the case of piecewise-linear activation functions with two saturated segments, but with smooth corners.
The existence of 3k equilibria for (1) is discussed in Section II. The global convergence of dynamics to 3k equilib-ria via sequential contracting is presented in Section III. More varieties of multiple equilibria are discussed in Section IV. The extension of the results in Sections II–IV to piecewise-linear activation functions is mentioned in Section V. We provide two numerical examples in Section VI. Finally, the conclusion is drawn in Section VII.
II. MULTIPLEEQUILIBRIA IN(1)
SetN = {1, 2, . . . , n}. The stationary equation for (1) is
Fi(x) := −μixi+ n
j=1
(αi j + βi j)gj(xj) + Ii = 0 (2) for i∈ N , and we denote F = (F1, . . . , Fn).
There are various ways to find the equilibrium for an ordi-nary differential equation system or delay equations. In this paper, we present an approach that combines a geometric
formulation on Fi(x) in (2) and the Brouwer’s fixed-point the-orem to study the existence of equilibrium for (1). Brouwer’s fixed-point theorem states as: every continuous function from a convex compact subset K of a Euclidean space to K itself has a fixed point. Our idea is to locate a region (box)
K := K1 × K2× · · · × Kn, with each Ki an interval in R, so that for an arbitrary (ζ1, . . . , ζn) ∈ K , there exists a
solution xi ∈ Ki to Fi(ζ1, . . . , ζi−1, xi, ζi+1, . . . , ζn) = 0, for
every i ∈ N . If this holds and the corresponding mapping (ζ1, . . . , ζn) = (x1, . . . , xn) is continuous, then there exists
a x that satisfies Fi(x) = 0 for all i ∈ N . x is then an equilibrium of (1). To locate such a region K , we develop a geometric formulation based on the structure of Fi.
Let us define the single-variable upper and lower functions ˆfi(ξ) := −μiξ + (αii+ βii)gi(ξ) + ki+
ˇfi(ξ) := −μiξ + (αii+ βii)gi(ξ) + ki−
for i ∈ N , where ki+ := nj=1, j=iρj|αi j + βi j| + Ii, ki−:= −nj=1, j=iρj|αi j + βi j| + Ii. It follows that
ˇfi(xi) ≤ Fi(x) ≤ ˆfi(xi)
for all x= (x1, . . . , xn) and i ∈ N . For Ji with ki−≤ Ji ≤ ki+, i ∈ N , we introduce a family of single-neuron equations
dξ
dt = fi(ξ) := −μiξ + (αii+ βii)gi(ξ) + Ji (3)
whereξ ∈ R.
For given parameters μi,αii,βii, and i∈ N , we denote M :=i ∈ N | max ξ∈Rg i(ξ) ≤ μi αii + βii B := i ∈ N | inf ξ∈Rg
i(ξ) < αiiμi+ β
ii < maxξ∈R gi(ξ)
. The notationM and B are related to the sense of monostable and (potentially) bistable scenarios, respectively.
Lemma 1: If i ∈ B, there exist two points ˜pi and ˜qi with ˜pi < σi < ˜qi such that fi( ˜pi) = fi( ˜qi) = 0, or equivalently, gi( ˜pi) = gi( ˜qi) = μi/(αii+ βii).
Proof: For each i ∈ N , with fi(ξ) = −μi + (αii + βii) gi(ξ), we have fi(ξ) = 0 if and only if gi(ξ) = μi/αii + βii. The graph of function gi(ξ) has a global maximum at σi and limξ→±∞gi(ξ) = 0. Hence, by continuity of gi, if
0= inf ξ∈Rg i(ξ) < μi αii+ βii < maxξ∈R gi(ξ) = gi(σi)
that is, and i ∈ B, there exist two points ˜pi and ˜qi, with ˜pi < σi < ˜qi, such that gi( ˜pi) = gi( ˜qi) = μi/αii+ βii.
Note that ˇfi, ˆfi, and fi are vertical shifts of one another, and they attain local minimum at ˜pi and local maximum at ˜qi.
We consider six disjoint subsets of B Br r = {i ∈ N |i ∈ B, ˇfi( ˜pi) > 0} Bl l = {i ∈ N|i ∈ B, ˆfi( ˜qi) < 0} B3 3 = {i ∈ N |i ∈ B, ˆfi( ˜pi) < 0, ˇfi( ˜qi) > 0} Br
3= {i ∈ N |i ∈ B, ˆfi( ˜pi) > 0, ˇfi( ˜pi) < 0, ˇfi( ˜qi) > 0} B3
l = {i ∈ N |i ∈ B, ˆfi( ˜pi) < 0, ˆfi( ˜qi) > 0, ˇfi( ˜qi) < 0} Br
l = {i ∈ N |i ∈ B, ˆfi( ˜pi) > 0, ˇfi( ˜qi) < 0}.
Fig. 1. (a)–(g) TypeM, Brr,Bll,B33,Br3,Bl3, andBlr, respectively.
Herein, l and r stand for left and right; the superscript and subscript • in B•,, • ∈ {l, r, 3} show the configurations for upper function ˆfi and lower function ˇfi, respectively. More precisely, if = l (resp., r), then ˆfi has a unique zero at the left (resp., right) arm of its graph; if = 3, then ˆfi has exactly three zeros. Same interpretation applies to• in B•and ˇfi. The configurations corresponding to eachB•are shown in Fig. 1. In particular, for i ∈ M ∪ Brr ∪ Bll, there exist points ˇmi, ˆmi with ˇmi < ˆmi such that ˆfi( ˆmi) = ˇfi( ˇmi) = 0. For i ∈ B33, there exist points ˆai, ˆbi, ˆci with ˆai < ˆbi < ˆci such that ˆfi(ˆai) = ˆfi( ˆbi) = ˆfi(ˆci) = 0 as well as points ˇai, ˇbi, ˇci with ˇai < ˇbi < ˇci, such that ˇfi(ˇai) = ˇfi( ˇbi) = ˇfi(ˇci) = 0, as shown in Fig. 1(d).
The following theorems are the main results of this section. We show that for each 1 ≤ k ≤ n, there exist parameters with which (1) admits 3k equilibria. Let card(•) denote the cardinality of set •.
Theorem 1: If M ∪ Brr ∪ Bll ∪ B33 = N and k= card(B33) ≥ 1, then there exist 3k equilibria in (1).
Proof: We consider 3k disjoint closed regions inRn ˜ w = (x 1, . . . , xn) ∈ Rn| xi ∈ ˜ wii (4) w= (w1, . . . , wn) wi = “l”, “m”, “r”, for i ∈ B3 3 wi = “s”, for i ∈ M ∪ Br r ∪ Bll
where ˜ li = [ˇai, ˆai], ˜ mi = [ ˆbi, ˇbi], ˜ ri = [ˇci, ˆci], and ˜ si = [ ˇmi, ˆmi] are compact intervals, as shown in Figs. 1 and 2. Let us take ˜ w as any one of these regions. For any given(ζ1, . . . , ζn) ∈ ˜ w, we solve for xi in
Fig. 2. (a) Graph of activation function gi in classA. (b) Configurations of functions ˆfiand ˇfi for i∈ B33.
where hi(xi) := −μixi + (αii + βii)gi(xi) + n
j=1, j=i (αi j + βi j)gj(ζj) + Ii, i ∈ N . Note that each hi is a vertical shift of fi in (3) and lies between ˆfi and ˇfi. Thus, for i ∈ B33, one can always find three solutions to (5), which lie in regions
˜ l i, ˜ m i , and ˜ r i, respectively. For i∈ M∪B r r∪Bll, there exists one solution to (5), which lies in region ˜ si. Next, we consider a mapping w: ˜ w→ ˜ w defined by
w(ζ
1, . . . , ζn) = ( ˜x1, . . . , ˜xn)
where ˜xi is the solution of (5) lying in ˜ wi i. The mapping w as defined is continuous, since each giis continuous. It follows from the Brouwer’s fixed-point theorem that there exists one fixed point ¯x = ( ¯x1, . . . , ¯xn) of win ˜ w, which is also a zero
of the function F, where F is defined in (2). Consequently, there exist 3k equilibria for (1) and each of the 3k disjoint regions ˜ w contains one of the equilibria.
We note that what was obtained in [30] is the existence of 3n equilibria for the n-neuron system (1). Theorem 1 establishes the existence of 3k equilibria for (1) via Brouwer’s fixed-point theorem. Next, we further apply the contraction mapping theorem to assert the existence of exact 3k equilibria for (1), under additional conditions. Let Li := gi(σi), i ∈ N .
Theorem 2: Assume that M ∪ Brr ∪ Bll ∪ B33 = N with k = card(B33) ≥ 1. For each i ∈ N , fix a θi ∈ (0, μi) and then define ¯Li := μ i−θi αii+βii, if i ∈ M ∪ B r r ∪ Bll Li, if i ∈ B33. (6) If the parameters satisfy
θi > n j=1, j=i ¯Lj|αi j+ βi j| (7) and gi(ξ) ⎧ ⎪ ⎨ ⎪ ⎩ < μi−θi αii+βii, ifξ ∈ [ ˇmi, ˆmi], i ∈ M ∪ B r r ∪ Bll < μi−θi αii+βii, ifξ ∈ (−∞, ˆai] ∪ [ˇci, ∞), i ∈ B 3 3 > μi+θi αii+βii, ifξ ∈ [ ˆbi, ˇbi], i ∈ B 3 3 (8)
for all i∈ N , then there exist exactly 3k equilibria in (1), and each region ˜ w, defined in (4), contains exactly one of these 3k equilibria.
Proof: Let ˜ w, w = (w1, . . . , wn), be any one of the
3k regions defined in (4). We shall show that w defined in Theorem 1 is a contraction map; there, hence, exists exactly one equilibrium lying in ˜ w. Assume that w(y) = y∗, w(x) = x∗, i.e., for each i = 1, . . . , n
⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ −μiyi∗+ (αii+ βii)gi(yi∗) + n
j=1, j=i(αi j+ βi j)gj(yj) +Ii = 0
−μixi∗+ (αii+ βii)gi(xi∗) + n
j=1, j=i(αi j+ βi j)gj(xj) +Ii = 0.
Then, subtracting one equation from the other, we obtain (xi∗− yi∗)[μi− (αii + βii)gi(ξi∗)] − n j=1, j=i (αi j+ βi j)g j(η∗j)[xj− yj] = 0
whereξi∗is some number between xi∗and yi∗, andη∗j is some number between xj and yj. Let us divide the discussion into four cases.
1) Ifwi = m, where i ∈ B33, then xi∗, yi∗, ξi∗∈ [ ˆbi, ˇbi], and gi(ξi∗) > (μi+ θi)/(αii + βii) by (8). Hence
|xi∗− yi∗| = n j=1, j=i (αi j+ βi j)gj(η∗j)(xj− yj) |(αii+ βii)gi(ξi∗) − μi| ≤ ⎧ ⎨ ⎩ ⎡ ⎣ n j=1, j=i ¯Lj|αi j+ βi j| ⎤ ⎦ /θi ⎫ ⎬ ⎭ x − y ∞ where[nj=1, j=i ¯Lj|αi j+ βi j|]/θi< 1, owing to (7). 2) If i ∈ M ∪ Br
r ∪ Bll, then xi∗, yi∗, ξi∗ ∈ [ ˇmi, ˆmi]. Thus, 0≤ gi(ξi∗) < (μi−θi)/(αii+βii), due to (8). It follows that |(αii+ βii)gi(ξi∗) − μi| = μi− (αii + βii)gi(ξi∗) > θi. Subsequently |xi∗− yi∗| ≤ ⎧ ⎨ ⎩ ⎡ ⎣ n j=1, j=i ¯Lj|αi j + βi j| ⎤ ⎦ |(αii+ βii)gi(ξi∗) − μi| ⎫ ⎬ ⎭ · x − y ∞ < ⎧ ⎨ ⎩ ⎡ ⎣ n j=1, j=i ¯Lj|αi j + βi j| ⎤ ⎦ θi ⎫ ⎬ ⎭ x − y ∞ where[nj=1, j=i ¯Lj|αi j+ βi j|]/θi< 1, owing to (7).
The situation for wi = r or l, i ∈ B33, is similar. Therefore, w is a contraction mapping and there exists a unique fixed point x = (x1, . . . , xn) of w, lying in ˜ w.
Restated, for each i = 1, . . . , n −μixi+ αiigi(xi) + n j=1, j=i αi jgj(xj) + n j=1 βi jgj(xj) + Ji = 0. (9) Thus, x is the unique equilibrium point of (1) lying in ˜ w. On the other hand, if x is an equilibrium point of (1), then its components satisfy (2), and thus must lie in some ˜ w. System (1), therefore, admits exactly 3k equilibria.
Notably, (8) describes how the slope of gi is smaller or larger than μi/(αii + βii) on the indicated regions with the help of θi. In fact, (7) prefers smaller |αi j + βi j|, i = j. Moreover, (7) favors largerθi ∈ (0, μi), since each ¯Lj in the right-hand side of (7) is lowered ifθj is larger, as seen from the definition of ¯Lj.
The existence of equilibrium for (1) in Theorem 1 is the most clear-cut situation, as for each i , the intersection scenario with the horizontal axis for the upper function ˆfi is identical to the one for the lower function ˇfi. We shall pursue more complicated cases where the upper and lower functions have disparate intersection configurations with the horizontal axis in Section IV.
III. GLOBALCONVERGENCE OFDYNAMICS IN(1) In this section, we shall discuss the globally convergent dynamics for (1), i.e., every solution of (1) converges to one of the equilibria. This implicates that almost every orbit of (1) converges to one of the stable equilibria. More precisely, the orbits, which do not lie on the stable manifolds of the unsta-ble equilibria, will converge to one of the staunsta-ble equilibria. To conclude globally convergent dynamics, we shall apply the newly developed sequential contracting scheme. We describe this scheme and formulation for (1) in Section III-A. We then implement the scheme to the case of 3k equilibria for the
n-neuron system in Section III-B. A. Sequential Contracting
An upper function and a lower function for each component of the equations were formulated to locate the equilibria in Section II. To study the dynamics, such a formulation is insufficient and it is tempting to design a sequence of upper and lower dynamics to capture the asymptotic behaviors of the solutions. This gives rise to the sequential contracting scheme. The idea is actually quite natural. We organize the coupling terms in (1) so that the dynamics corresponding to each component equation can be controlled by some scalar equations. We start by constructing a preliminary upper equa-tion and a lower equaequa-tion for each component. Such upper and lower dynamics can usually be constructed due to the dissipative property of the coupled systems. The upper and lower equations are designed to have their own dynamics contracted to some compact intervals. Each component for the original dynamics of (1) is then trapped in the interval
after certain time. We then construct finer upper and lower dynamics, so that the original dynamics are attracted to even more concentrated regions after a later time. We then formulate a criterion for contraction under, which these nested intervals collapse into points, as the iterative constructions of upper and lower dynamics carry on continuously.
Notably, (1) is dissipative, as observed from the equations that the summation terms in (1) are bounded. Such a property was formally justified in [18]. Therefore, a solution x(t) = x(t; t0, φ) of (1), starting from arbitrary
φ ∈ C([−τM, 0], Rn) at t = t
0, exists on [t0, ∞).
In the following discussion, we fix an initial condition φ. Its evolution x(t) = (x1(t), . . . , xn(t)) is then a fixed function
defined on[t0, ∞) and the ith component xi(t) satisfies
˙xi(t) = −μixi(t) + αiigi(xi(t)) + βiigi(xi(t − τii)) + wi (10) for all t ≥ t0, where wi :=
j=i{αi jgj(xj(t)) + βi jgj(xj(t −τi j))} + Ii. Herein, eachwi, i∈ N , is a function of t, which is defined from the solution x(t); i.e., wi varies with respect to solutions. For a fixed solution x(t), wiis indeed a function of t, i.e., wi = wi(t), and (10) is like a scalar equation for each i .
For later use, we define for each i ∈ N wmax i (∞) := lim T→∞w max i (T ), wmini (∞) := lim T→∞w min i (T ) wherewmaxi (T ) := sup{wi(t) | t ≥ T } is nonincreasing in T and wmini (T ) := inf{wi(t) | t ≥ T } is nondecreasing in T . We define the following preliminary upper and lower bounds for (10), respectively:
ˆhi(ξ) := −μiξ + 2(|αii| + |βii|)ρi + wmaxi (t0) (11)
ˇhi(ξ) := −μiξ − 2(|αii| + |βii|)ρi + wmini (t0) (12)
where ˆhi and ˇhi are linear decreasing functions with unique zeros: ˆAhi and ˇAhi, where ˆAhi := [2(|αii| + |βii|)ρi + wmax
i (t0)]/μi and ˇAhi := [−2(|αii| + |βii|)ρi + wmini (t0)]/μi,
respectively. Notably, ˆhi( ˇAhi) = − ˇhi( ˆAhi) = 4(|αii|+|βii|)ρi+ wmax
i (t0) − w
min
i (t0) ≥ 0. Applying the arguments similar to those for [33, Lemma 2.1] reveals that for each i ∈ N
ˇhi(xi(t)) + (|αii| + |βii|)ρi
≤ ˙xi(t) ≤ ˆhi(xi(t)) − (|αii| + |βii|)ρi for all t ≥ t0. Consequently, there exists a tφ (depending on
φ) such that xi(t) enters and remains in interval [ ˇAh
i, ˆAhi] for all i and t ≥ tφ. Accordingly, we can construct the second preliminary upper and lower bounds for (10)
ˆf(0) i (ξ, T ) := ˆγi(ξ, T ) − βiiLiτiiˇhi ˆAhi if βii ≥ 0 ˆγi(ξ, T ) − βiiLiτiiˆhi ˇAhi if βii < 0 (13) ˇf(0) i (ξ, T ) := ˇγi(ξ, T ) − βiiLiτiiˆhi ˇAhi if βii ≥ 0 ˇγi(ξ, T ) − βiiLiτiiˇhi ˆAhi if βii < 0 (14)
where
ˆγi(ξ, T ) := −μiξ + (αii+ βii)gi(ξ) + wmaxi (T ) ˇγi(ξ, T ) := −μiξ + (αii+ βii)gi(ξ) + wmini (T ) for some T ≥ t0. It is not difficult to see that condition (M1)
(in the Appendix) implies|αii| + |βii| > 0, and thus ˇhi(ξ) < ˇfi(0)(ξ, t0) ≤ ˇfi(0)(ξ, T )
≤ ˆfi(0)(ξ, T ) ≤ ˆfi(0)(ξ, t0) < ˆhi(ξ) (15) for all T ≥ t0andξ ∈ R. From (15), conditions (M1) and (M2)
(in the Appendix) imply that there exists a unique zero ˇm(0)i (T ) [resp., ˆm(0)i (T )] of ˇfi(0)(·, T ) = 0 [resp., ˆfi(0)(·, T ) = 0] for each T ≥ T0, where T0is defined in condition (M2). Moreover
ˇAh
i < ˇm(0)i (T0) ≤ ˇm(0)i (T ) ≤ ˆm(0)i (T ) ≤ ˆm(0)i (T0) < ˆA h i. By arguments similar to those for [33, Lemma 2.2], we can show that for any T ≥ max{tφ+ τii, T0}
ˇf(0)
i (xi(t), T ) + |βii|(|αii| + |βii|)ρiLiτii
≤ ˙xi(t) ≤ ˆfi(0)(xi(t), T ) − |βii|(|αii| + |βii|)ρiLiτii for t ≥ T . Consequently, xi(t) enters and remains in interval [ ˇm(0)i (T ), ˆm(0)i (T )] contained in [ ˇA
h i, ˆA
h
i] after certain time. Then, iteratively applying arguments based on constructing finer upper ˆfi(k) and lower bounds ˇfi(k) for (10) allows us to establish the convergence of xi(t) to some compact interval, say Ji, as t → ∞. The formulation of such ˆfi(k) and ˇfi(k) and the precise statement of this convergence (Proposition 2) are arranged in the Appendix. Herein, we say that a real-valued function y(t) converges to a compact interval J if
d(y(t), J) := inf{|y(t) − ζ | : ζ ∈ J} → 0, as t → ∞. On the
other hand, another possible configuration of upper and lower functions leads to the convergence of xi(t) to one of three compact intervals, which is summarized in Proposition 3 (in the Appendix). As performed component by component successively, this sequential contracting scheme converts the convergence of solution x(t) to an equilibrium of (1) into solving a corresponding linear system of algebraic equations via the following proposition.
Proposition 1: Let x(t) = (x1(t), . . . , xn(t)) be a fixed
solution of (1). Assume that for every i ∈ N , there exists a compact interval Ji of length di, such that xi(t) converges to Ji and di satisfies di ≤ wmax i (∞) − w min i (∞) ηi
for someηi > 0, and there exist a compact interval ˜Ji and a ˜Li ≥ 0, such that Ji ⊆ ˜Ji and
gi(ξ) ≤ ˜Li for allξ ∈ ˜Ji.
Let M := [mi j]1≤i, j≤n with mii := ηi, mi j := −(|αi j| + |βi j|) ˜Lj for i = j. If the Gauss–Seidel iteration for solving the linear system
Mv= 0 (16)
converges to zero, the unique solution of (16), or equivalently, ˜λM < 1, then every di degenerates into zero, and the solution
x(t) of (1) converges to a singleton, where ˜λM := max
1≤i≤n{|λi| : λi : eigenvalue of (DM− LM) −1U
M} and M = DM− LM− UM with DM, −LM, and −UM the diagonal, strictly lower triangular and strictly upper triangular parts of M, respectively.
Proof: We first claim that for each i ∈ N , there exists a
sequence of compact intervals {Ji(k)}∞k=0 with Ji(k) ⊇ Ji, and the length di(k) of Ji(k) satisfies
di(k) = ⎧ ⎨ ⎩ i−1 j=1 (|αi j| + |βi j|) ˜Ljd(k)j + n j=i+1 (|αi j| + |βi j|) ˜Ljd(k−1)j ⎫ ⎬ ⎭ ηi (17) for every k ∈ N, where Ji(0) = ˜Ji. Let us sketch the arguments for the claim through induction. Assume that the claim holds for k= ˜k − 1 ∈ N and all i ∈ N for some ˜k ≥ 2, and it holds for k = ˜k and every i ∈ {1, . . . , − 1} with 1≤ −1 < n. That is, xi(t) converges to the compact interval Ji(˜k)(resp., Ji(˜k−1)) if i = 1, . . . , −1 (resp., i = +1, . . . , n). Set ˇ W(˜k)(∞) := I+ −1 j=1 min ξ,η∈J(˜k)j ∩ ˜Jj {αjgj(ξ) + βjgj(η)} + n j=+1 min ξ,η∈J(˜k−1)j ∩ ˜Jj {αjgj(ξ) + βjgj(η)} ˆ W(˜k)(∞) := I+ −1 j=1 max ξ,η∈J(˜k)j ∩ ˜Jj {αjgj(ξ) + βjgj(η)} + n j=+1 max ξ,η∈J(˜k−1)j ∩ ˜Jj {αjgj(ξ) + βjgj(η)}. In respecting the term w associated with the solution (x1(t), . . . , xn(t)) defined in (10), and the definition of
wmin
(∞) and wmax(∞), we obtain
ˇ
W(˜k)(∞) ≤ wmin (∞) ≤ wmax (∞) ≤ ˆW(˜k)(∞)
noting that xi(t) converges to Ji(˜k)∩ ˜Ji (resp., Ji(˜k−1)∩ ˜Ji) if i = 1, . . . , − 1 (resp., i = + 1, . . . , n). Thus, x(t) converges to J with its length d satisfying
d ≤wmax (∞) − wmin(∞)η ≤ ˆW(˜k)(∞) − ˇW(˜k)(∞)η ≤ ⎧ ⎨ ⎩ −1 j=1 (|αj| + |βj|) ˜Ljd(˜k)j + n j=+1 (|αj| + |βj|) ˜Ljd(˜k−1)j ⎫ ⎬ ⎭ η= d(˜k)
where the last inequality follows from applying the mean valve theorem to ˆW(˜k)(∞) − ˇW(˜k)(∞), recalling gi(ξ) ≤ ˜Li if ξ ∈ Ji(k) ∩ ˜Ji and Ji(k) ∩ ˜Ji ⊆ ˜Ji, for all i ∈ N and k∈ N. These computations also explain the formulation of di(k)
in (17). Subsequently, x(t) converges to a compact interval
J(˜k) that contains J and whose length is exactly d(˜k). That is, the assertion holds for k = ˜k and i = as well. We thus justify the claim.
We observe that {(d1(k), . . . , dn(k))}k∈N is exactly the Gauss–Seidel iteration for solving the linear system (16). From the previous arguments, it follows that(x1(t), . . . , xn(t))
converges to a singleton if the iteration {(d1(k), . . . , dn(k))}k∈N satisfies di(k) → 0 as k → ∞, for all i ∈ N . In addition, ˜λM < 1 is a sufficient and necessary condition for such a convergence of the Gauss–Seidel iteration for (16) [15].
The convergence to zero for Gauss–Seidel iteration in Proposition 1 can be assured if M is strictly diagonal dominant [37].
Lemma 2: Under the strictly diagonal dominance of
M: ηi > j=i{(|αi j| + |βi j|) ˜Lj} for all i ∈ N , the Gauss–Seidel iteration converges to zero, the unique solution of (16).
B. Global Convergence to 3k Equilibria for (1)
Next, we investigate the convergence of dynamics in (1) with 3k equilibria. Notice that the formulation of ˇhi, ˆhi in (11) and (12), and ˆfi(0)(·, T ), ˇfi(0)(·, T ) in (13) and (14),
depends on a given solution x(t). To formulate the convergence condition for all solutions of (1), we need to employ the following solution-independent upper and lower functions:
ˆFi(ξ) = ˆi(ξ) + ˜ki+, ˇFi(ξ) = ˇi(ξ) + ˜ki− (18) where ˜k+i := nj=1, j=i(|αi j| + |βi j|)ρj + Ii,
˜k− i := −
n
j=1, j=i(|αi j| + |βi j|)ρj + Ii, and ˆi(ξ)
:=
−μiξ + (αii + βii)gi(ξ) − βiiLiτii ˇHi( ˆAHi ) if βii ≥ 0 −μiξ + (αii + βii)gi(ξ) − βiiLiτii ˆHi( ˇAHi ) if βii < 0 ˇi(ξ)
:=
−μiξ + (αii + βii)gi(ξ) − βiiLiτii ˆHi( ˇAHi ) if βii ≥ 0 −μiξ + (αii + βii)gi(ξ) − βiiLiτii ˇHi( ˆAHi ) if βii < 0 and ˆAiH and ˇAiH are, respectively, the unique zeros of the linear decreasing functions ˆHi and ˇHi defined as
ˆ
Hi(ξ) := −μiξ + 2(|αii| + |βii|)ρi+ ˜k+i ˇ
Hi(ξ) := −μiξ − 2(|αii| + |βii|)ρi+ ˜k−i .
ˆFi and ˇFi are also vertical shifts of ˆfi and ˇfi and ˆfi(0)(·, T ), ˇf(0)
i (·, T ) defined in Sections II and III-A, respectively. It is not difficult to verify
ˇ
Hi(ξ) ≤ ˇhi(ξ) ≤ ˆhi(ξ) ≤ ˆHi(ξ) (19) ˇFi(ξ) ≤ ˇfi(0)(ξ, T ) ≤ ˆfi(0)(ξ, T ) ≤ ˆFi(ξ) (20)
for all T ≥ t0 andξ ∈ R. Moreover
ˇFi(ξ) ≤ ˇfi(ξ) ≤ ˆfi(ξ) ≤ ˆFi(ξ) (21) for allξ ∈ R, as ˆHi( ˇAiH) ≥ 0 and ˇHi( ˆAiH) ≤ 0.
WhenM ∪ Brr∪ Bll∪ B33= N holds, we further consider the condition ⎧ ⎨ ⎩ ˇFi( ˜pi) > 0 if i ∈ Brr ˆFi( ˜qi) < 0 if i ∈ Bll ˆFi( ˜pi) < 0, ˇFi( ˜qi) > 0 if i ∈ B33
(22) where ˜pi, ˜qi are defined in Lemma 1. Under (22), there exists a unique zero ˇmiF (resp., ˆmiF) to ˇFi (resp., ˆFi), if i ∈ M ∪ Brr ∪ Bll, and there exist exactly three zeros ˆaiF,
ˆbF
i , ˆciF (resp., ˇaiF, ˇbiF, ˇciF) to ˆFi (resp., ˇFi), if i ∈ B33, and
ˇaF i ≤ ˆa F i < ˜pi < ˆb F i ≤ ˇb F i < ˜qi < ˇc F i ≤ ˆc F i . Let τc ii := (|αii| + |βii|)ρi Li[4(|αii| + |βii|)ρi + 2 n j=1, j=i(|αi j| + |βi j|)ρj]. To conclude the stability of 2k out of these 3k equilibria, we further need the following functions:
ˆIi(ξ) := −μiξ + αiigi(ξ) + n j=1, j=i |αi j|ρj+ n j=1 |βi j|ρj+ Ii ˇIi(ξ) := −μiξ + αiigi(ξ) −
n j=1,=i |αi j|ρj− n j=1 |βi j|ρj+ Ii for i∈ N . Notably ˇIi(ξ) ≤ ˇfi(ξ) < ˆfi(ξ) ≤ ˆIi(ξ) (23) for all ξ ∈ R. Moreover, if Li > μi/αii > 0, there exist exactly two points ¯pi and ¯qi with ¯pi < σi < ¯qi such that gi( ¯pi) = gi( ¯qi) = μi/αii. If in addition that ˇIi( ¯qi) > 0 and ˆIi( ¯pi) < 0, then there exist exactly three zeros ˆaI
i, ˆbiI, and ˆciI (resp., ˇaiI, ˇbiI, and ˇciI) for ˆIi (resp., ˇIi), where ˇaI i ≤ ˆa I i < ¯pi < ˆb I i ≤ ˇb I i < ¯qi < ˇc I i ≤ ˆc I i.
We assumeB33= ∅ and consider the following condition.
Condition (I): Li > μi/αii > 0, ˇIi( ¯qi) > 0, and ˆIi( ¯pi) < 0, for i∈ B33.
AssumingM ∪ Brr∪ Bll∪ B33= N with card(B33) = k ≥ 1, we define the following 2k regions:
¯ w = {(x 1, . . . , xn) ∈ Rn| xi ∈ ¯ wii} (24) w= (w1, . . . , wn) wi = “l”, “r”, for i ∈ B3 3 wi = “s”, for i ∈ M ∪ Br r ∪ Bll
where ¯ li = (−∞, ˆbiI), ¯ ri = ( ˇbiI, ∞), and ¯ si = R, under condition (I). Notably, ˜ w ⊆ ¯ wand ˜ w∩ ¯ w = ∅ if w = w due to (23). Denote by¯xwthe equilibrium lying in ¯ w. We can then establish the global convergence of dynamics for (1) and show that each equilibrium ¯xw is attracting in the sense that every solution evolved from initial value in ¯ w converges to ¯xw.
Theorem 3: Assume that M ∪ Brr ∪ Bll ∪ B33 = N ,
(7) and (22) hold, and for each i ∈ N
and gi(ξ) ⎧ ⎪ ⎨ ⎪ ⎩ < μi−θi αii+βii, if ξ ∈ ˇmF i , ˆm F i , i ∈ M ∪ Brr ∪ Bll < μi−θi αii+βii, if ξ ∈ − ∞, ˆaF i ∪ˇcF i , ∞ , i ∈ B33 > μi+θi αii+βii, if ξ ∈ ˆb F i , ˇb F i , i∈ B33 (26) for someθi ∈ (0, μi). Then, (1) achieves global convergence to the 3k equilibria provided that the Gauss–Seidel iteration for the linear algebraic system (16) converges to zero, the unique solution, where mii = (1 − 2|βii|Liτii)θi for i ∈ N , mi j = −(|αi j| + |βi j|) ¯Lj for i, j ∈ N , i = j, and ¯Lj is defined in (6). If condition (I) holds in addition, then the 2k out of these 3k equilibria are attracting; more precisely, every solution x(t) evolved from an initial value in ¯ w converges to ¯xw, as t→ ∞, where ¯ w is defined in (24).
Proof: Based on (21), it can be verified that (26) implies (8). Consequently, (1) admits exactly 3k equilibria under the conditions imposed by Theorem 2. Next, let us prove the convergence of dynamics. Let (x1(t), . . . , xn(t)) be an arbitrary solution of (1). Then,
each component ˙xi(t) can be written in the form (10). If i ∈ M ∪ Brr ∪ Bll, then there exists a unique zero
ˇmf i (resp., ˆm f i ) to ˇfi(0)(·, t0) [resp., ˆfi(0)(·, t0)] where [ ˇmf i , ˆm f
i ] ⊆ [ ˇmFi , ˆmiF], by (20) and the first two inequalities in (22). If i ∈ B33, then there exist exactly three zeros ˆaif, ˆbif, and ˆcif (resp., ˇaif, ˇbif, and ˇcif) to ˆfi(0)(·, t0) [resp., ˇfi(0)(·, t0)]
by the last inequality in (22); in addition,[ˇaif, ˆaif] ⊆ [ˇaiF, ˆaiF], [ ˆbf i , ˇb f i ] ⊆ [ ˆbiF, ˇbiF], and [ˇc f i, ˆc f i ] ⊆ [ˇciF, ˆciF]. Then, it is not difficult to verify that component xi(t) satisfies (M1)–(M3) if i ∈ M ∪ Brr ∪ Bll, and satisfies (B1)–(B3) for i ∈ B33, under (22), (25), and (26) (see the Appendix). By Propositions 2 and 3, we obtain the convergence of xi(t) to an interval Ji of length di for every i ∈ N , where di satisfies di ≤ wmax i (∞) − wmini (∞)
/[(1 − 2|βii|Liτii)θi]. (27) Notably, Ji ⊆ [ ˇmFi , ˆmiF] if i ∈ M ∪ Brr ∪ Bll, and Ji ⊆ [ˇaiF, ˆaiF] ∪ [ ˆbiF, ˇbiF] ∪ [ˇciF, ˆciF] ⊆ [ˇaFj, ˆcFj] if i ∈ B33. If i ∈ B33, which of the intervals[ˇaiF, ˆaiF], [ ˆbiF, ˇbiF], [ˇciF, ˆciF]
Ji is actually contained in depends on the initial condition; the detailed arguments are similar to [33, Proposition 4]. Moreover, gj(ξ) ≤ ¯Lj, where ¯Lj = μ j−θj αj j+βj j if j∈ M ∪ B r r ∪ Bll,ξ ∈ ˇmF j, ˆmFj Lj if j∈ B33,ξ ∈ ˇaF j , ˆcFj .
By Proposition 1, it follows that (x1(t), . . . , xn(t)) converges
to a singleton if the Gauss–Seidel iteration for (16) converges to zero, the unique solution of (16), where mii = (1 − 2|βii| Liτii)θi if i ∈ N and mi j = −(|αi j| + |βi j|) ¯Lj if i, j ∈ N and i = j.
Below, let us show that ¯ w is positively invariant under the solution flow of (1). Let (x1(t), . . . , xn(t)) be a solution
evolved from an initial condition lying ¯ w. From (1), we obtain
ˇIi(xi(t)) ≤ ˙xi(t) ≤ ˆIi(xi(t)). (28)
From (28) and the configurations of ˇIi and ˆIi, for i ∈ B33, we see that ˙xi(t) < 0, should xi(t) stay in (ˆaiI, ˆbiI) ∪ (ˆciI, ∞), and ˙xi(t) > 0, should xi(t) stay in (−∞, ˇaiI) ∪ ( ˇbiI, ˇciI) under condition (I). Consequently, for i ∈ B33, xi(t) remains in (−∞, ˆbI
i) [resp., ( ˇbiI, ∞)] and actually converges to [ˇaiI, ˆaiI] (resp.,[ˇciI, ˆciI]) if wi = l (resp., r). We hence justify the pos-itive invariance of ¯ w. Subsequently, every solution evolved from an initial value in ¯ wconverges to ¯xw, due to the global convergence of (1).
Corollary 1: If M is strictly diagonal-dominant, i.e., (1 − 2|βii|Liτii)θi > j=i{(|αi j| + |βi j|) ¯Lj}, for all i ∈ N , then the Gauss–Seidel iteration converges to zero, and the assertions of Theorem 3 hold under the same assumptions.
Remark 1: 1) In the proof of Theorem 3, it was shown
that every solution converges to a single point as t → ∞. This point is certainly an equilibrium of (1). 2) Theorem 3 concludes that the 2k equilibria out of these 3k equilibria are attracting in (1). By applying the arguments in [30, Th. 3.4], we can further prove that these 2k equilibria are asymptotically stable and the other(3k− 2k) equilibria are unstable without additional assumptions.
We stress that to establish analytic theory to conclude the global dynamics for (1), several additional conditions have been imposed in Theorem 3. These are certainly sufficient conditions that are formulated according to the mathemati-cal methodologies. It is likely that the convergent dynamics already holds under the assumption of Theorem 1.
IV. OTHERCASES OFMULTISTABILITY
In this section, we elaborate on the other cases of multiple equilibria in (1). These cases are more complicated than the one in Section II. We illustrate the idea for n= 2.
System (1) with n= 2 reads as
d x1(t) dt = −μ1x1(t) + α11g1(x1(t)) + α12g2(x2(t)) + β11g1(x1(t − τ11)) + β12g2(x2(t − τ12)) + I1 (29) d x2(t) dt = −μ2x2(t) + α21g1(x1(t)) + α22g2(x2(t)) + β21g1(x1(t − τ21)) + β22g2(x2(t − τ22)) + I2. (30) The upper and lower functions are now
ˆfi(ξ) = −μiξ + (αii + βii)gi(ξ) + |αi j+ βi j|ρj+ Ii ˇfi(ξ) = −μiξ + (αii + βii)gi(ξ) − |αi j+ βi j|ρj+ Ii where i, j ∈ {1, 2} and j = i. For this two-neuron system, there are four basic types: 1)(M, M); 2) (M, B); 3) (B, M); and 4) (B, B), which correspond to 1, 2 ∈ M, 1 ∈ M and 2∈ B, 1 ∈ B and 2 ∈ M, and 1, 2 ∈ B, respectively, according to the notation in Section II. We further denote the following subtypes of these four types.
Notation 1: Denote subtype (mm)(mm) if 1, 2 ∈ M, (mm)(•)
if 1 ∈ M and 2 ∈ B•, (•)(mm) if 1 ∈ B• and 2 ∈ M, and (•)(•) if 1 ∈ B•and 2∈ B•.
TABLE I
SUBTYPES IN(M, M), (M, B), (B, M),AND(B, B)
In these notations, the first (second) column corresponds to the configurations of upper and lower functions for the first neuron
i = 1 (second neuron i = 2). The notation B• is as defined in Section II. We list these 20 subtypes Tk, k = 1, . . . , 20, in Table I. The cases of T1, T2, T3, T6, T7, T10, T11, and T14
have been discussed in Section II.
We need the following notation to further describe the geometric configurations of the upper and lower functions.
Notation 2:
1) For an interval I = [, ς], we say that I > 0 (< 0) if > 0 (ς < 0). Denote cI := [c, cς], if
c ≥ 0, cI := [cς, c], if c < 0; β + I := [β + , β + ς], for a real number β; gi(I) := [gi(), gi(ς)] for increasing function gi.
2) For intervalsIk= [k, ςk], k = 1, 2, I1+I2:= [1+2,
ς1+ ς2], an interval.
3) For a givenη ∈ R, define functions fiη(ξ) := −μiξ + (αii+βii)gi(ξ)+(αi j+βi j)gj(η)+Ii, where i, j ∈ {1, 2} and j = i.
4) For each ξ, define intervals Ki(ξ; I) := −μiξ + (αii+βii)gi(ξ)+(αi j+βi j)gj(I)+Ii = { fiη(ξ) : η ∈ I}, where i, j ∈ {1, 2}, j = i. Regarding ξ as a variable, denote by Ki(ξ; [, ς]) an interval-valued function of ξ. Note that the graph of Ki(ξ; [, ς]) is a strip that lies between the graphs of functions fi(ξ) and fiς(ξ).
Herein, we demonstrate the analysis for the existence of equilibria in three cases: 1)(rr)(r3) ∈ T12; 2)(33)(r3) ∈ T15; and
3)(33)(rl) ∈ T16. First, let us take the case(rr)(r3) to introduce
further notation. In this case, we have ˇf1( ˇm1) = ˆf1( ˆm1) = 0
with ˇm1 < ˆm1. Denote S1 := [ ˇm1, ˆm1], an interval.
If K2( ˜p2; S1) < 0, the strip between the graphs of functions f ˇm1
2 and f2ˆm1 intersects the horizontal axis by three intervals.
Here, we take α21 + β21 > 0 to introduce the following
notation and the case of opposite sign can be treated similarly.
Fig. 3. Configuration underα21+ β21> 0. (a) Graphs of ˆf1(ξ) and ˇf1(ξ). (b) K2( ˜p2; S1) < 0: strip K2(ξ; S1) bounded by the graphs of f2ˆm1 and
fˇm1
2 intersectsξ-axis by three intervals. (c) K2( ˜p2; S1) > 0: strip K2(ξ; S1) intersectsξ-axis by one interval.
Each of functions f ˇm1
2 and f ˆm
1
2 has three zeros,
say f ˇm1
2 (ˇa2(1)) = f2ˇm1( ˇb(1)2 ) = f2ˇm1(ˇc2(1)) = 0, and f ˆm1
2 (ˆa(1)2 ) = f2ˆm1( ˆb2(1)) = f2ˆm1(ˆc(1)2 ) = 0 with
ˇa(1)2 < ˆa2(1) < ˆb2(1) < ˇb2(1) < ˇc2(1) < ˆc2(1) [Fig. 3(a) and (b)].
Denote the intervals AS1
2 := [ˇa (1) 2 , ˆa (1) 2 ], B S1 2 := [ ˆb (1) 2 , ˇb (1) 2 ], and CS1
2 := [ˇc(1)2 , ˆc(1)2 ]. Let us explain the notation: A
S1
2 is
obtained by the intersections of the left arms of f ˇm1
2 and f2ˆm1
with the horizontal axis; in addition, the values of F2(x1, x2)
lie between f ˇm1
2 (x2) and f2ˆm1(x2), when x1is restricted to S1.
Similar interpretation applies to BS1
2 and C
S1
2 .
If K2( ˜p2; S1) > 0, then each of functions f2ˇm1(ξ) and f ˆm1
2 (ξ) has one zero, say f2ˇm1( ˇm(1)2 ) = 0 and f2ˆm1( ˆm(1)2 ) = 0
with ˇm(1)2 < ˆm(1)2 [Fig. 3(c)]. Denote the interval
SS1
2 = [ ˇm(1)2 , ˆm(1)2 ]. In this case, the strip between the graphs
of f ˇm1
2 and f ˆm
1
2 intersects the horizontal axis by one interval. Theorem 4: Consider (29) and (30) with the case (rr)(r3).
There exists one equilibrium if K2( ˜p2; S1) > 0, and three
equilibria if K2( ˜p2; S1) < 0.
Proof: For the case K2( ˜p2; S1) < 0, we consider the
following three regions: w := (x
1, x2) ∈ R2|xi ∈ wii w = (w1, w2), w1= s, w2= l, m, r
where s1 = S1, l2 = A2S1, m2 = B2S1, and r2 = C2S1. Let
wbe any one of these regions. For any given(ζ
1, ζ2) ∈ w,
we solve for xi in
−μixi + (αii+ βii)gi(xi) + (αi j+ βi j)gj(ζj) + Ii = 0 (31) for i, j ∈ {1, 2} and j = i. Note that ˇf1(ξ) ≤ f1η(ξ) ≤ ˆf1(ξ)
for all η ∈ w2 2 and f ˇm 1 2 (ξ) ≤ f2η(ξ) ≤ f ˆm 1 2 (ξ) [resp., f ˆm1 2 (ξ) ≤ f η 2(ξ) ≤ f ˇm 1 2 (ξ)] for all η ∈ S1 if α21+ β21 > 0
(resp., α21 + β21 < 0). Accordingly, one can always find
three solutions to (31), which lie in regions li, mi , and r
i, respectively. We define a mapping w : w → w by w(ζ1, ζ2) = ( ˜x1, ˜x2) where ˜xi is the solution of (31).
The mapping w as defined is continuous, as in the proof of Theorem 1. It follows from the Brouwer’s fixed-point theorem
TABLE II
CRITERIA FORVARIOUSNUMBERS OFEQUILIBRIUM
POINTS FOR THECASE(33)(r3)
TABLE III
CRITERIA FORVARIOUSNUMBERS OFEQUILIBRIUM
POINTS FOR THECASE(33)(rl)
that there exists a fixed point ¯x = ( ¯x1, ¯x2) of w in w,
which is also a zero of F in (2). Consequently, there exist three equilibria for (29) and each region w contains one equilibrium.
If K2( ˜p2; S1) > 0, then by constructing a continuous
mapping on the region w := S1 × S2S1, where SS1 2 := [ ˇm(1)2 , ˆm(1)2 ] and ˇf2ˇm1( ˇm(1)2 ) = ˆf2ˆm1( ˆm(1)2 ) = 0 [resp., ˇfˆm1 2 ( ˇm (1) 2 ) = ˆfˇm 2 2 ( ˆm (1) 2 ) = 0] if α21 + β21 > 0
(resp.,< 0), and by similar arguments, we conclude that there exists an equilibrium for (29), which lies in the region w.
Denote A1 := [ˇa1, ˆa1], B1:= [ ˆb1, ˇb1], and C1 := [ˇc1, ˆc1].
By similar arguments, we can obtain the following existence of multiple equilibria.
Theorem 5: Consider the case(33)(r3) or (33)(rl). Then, there
exist parameters such that (29) and (30) have three, five, seven, or nine equilibria. The criterion for each of these existence is listed in Tables II and III, respectively.
Proof: We only show the case (33)(r3). The other cases
are similar.
If α21+ β21> 0 and K2( ˜p2; A1) > 0 or if α21+ β21< 0
and K2( ˜p2; C1) > 0, then K2(ξ; •) intersects the ξ-axis by
one interval, say S2•, for • = A1, B1, and C1. Thus, there
exist three regions: A1× S2A1, B1× S2B1, and C1× S2C1, and
each can be regarded as region w in the proof of Theorem 4; hence, there are three equilibria.
If α21+ β21 > 0, K2( ˜p2; A1) < 0, and K2( ˜p2; B1) > 0,
then K2(ξ; A1) intersects the ξ-axis by three intervals,
say AA1
2 , B
A1
2 , C
A1
2 , and K2(ξ; •) intersects the ξ-axis by
one interval, say S2•, for • = B1, C1. Then, there exist
five regions A1× A2A1, A1× B2A1, A1× C2A1, B1× S2B1, and C1× S2C1 and hence five equilibria. Same assertion holds if
α21+ β21 < 0, K2( ˜p2; C1) < 0, and K2( ˜p2; B1) > 0.
Ifα21+ β21 > 0, K2( ˜p2; B1) < 0, and K2( ˜p2; C1) > 0 or
α21+ β21 < 0, K2( ˜p2; B1) < 0, and K2( ˜p2; A1) > 0, there
are seven regions to be considered as previous cases and hence seven equilibria.
If α21+ β21 > 0, K2( ˜p2; C1) < 0 or α21+ β21 < 0, and K2( ˜p2; A1) < 0, there are nine regions to be considered and
hence nine equilibria.
We can apply sequential contracting formulation similar to Theorem 3 to obtain global convergence of dynamics for the cases of multiple equilibria in Theorems 4 and 5. As the above analysis is performed componentwise, it certainly can be extended to the general n-neuron system (1).
V. PIECEWISE-LINEARACTIVATIONFUNCTIONS
It is straightforward to extend the discussions in Sections II–IV to (1) with piecewise-linear activation functions gi(ξ) = ⎧ ⎨ ⎩ ui if − ∞ < ξ < ¯pi ui +v¯qi−ui i− ¯pi(ξ − ¯pi) if ¯pi ≤ ξ ≤ ¯qi vi if ¯qi < ξ < ∞. (32) This class of functions includes the standard activation function in cellular neural networks
gi(ξ) = 1
2(|ξ + 1| − |ξ − 1|), i ∈ N . In this case, we adapt the settingM and B to
M := i ∈ N | vi− ui ¯qi− ¯pi ≤ μi αii + βii B := i ∈ N | 0 < μi αii + βii < vi − ui ¯qi− ¯pi .
Further classification according to the intersections of ˆfi and ˇfi with the horizontal axis, as those shown in Fig. 1, can be formulated with ˜pi and ˜qi replaced by ¯pi and ¯qi, respectively. All the results in Sections II–IV can thus be extended to (1) with such piecewise-linear activation functions. The extension also holds if the middle part of gi for ¯pi ≤ ξ ≤ ¯qi in (32) is replaced by an increasing function connecting the two saturation segments.
VI. NUMERICALILLUSTRATIONS
In this section, we present two examples for (1) with n= 2 or n= 3 to illustrate the theorems in Sections II–IV. We take the activation function as gi(ξ) = tanh(ξ), and thus σi = 0, ρi = 1, Li = 1 for i = 1, 2, 3.
Example 1: This example illustrates Theorem 5 for the case n= 2 and (33)(r3). We consider the parameters
μ1= 1, α11= 1.5, α12 = 0.07, β11= 0.1, β12 = 0.08
μ2= 1, α21= 0.1, α22= 1.4, β21= 0.1, β22 = 0.1 I1= −0.05, I2= 0.32
Fig. 4. Numerical simulation for Example 1.
Herein, α12 + β12 = 0.15 > 0, α21 + β21 = 0.2 > 0.
Moreover, ˜p1 = −0.7127085, ˜q1 = 0.7127085, ˜p2 =
−0.6584789, ˜q2 = 0.6584789, k1+ = 0.1, k−1 = −0.2, k2+ = 0.52, and k−2 = 0.12. Let us examine the
condition in Theorem 5. We compute to find intervals
A1 := [ˇa1, ˆa1] = [−1.69581395, −1.26304012] and B1 := [ ˆb1, ˇb1] = [−0.17106488, 0.37914872]. Condition K2( ˜p2; A1) < 0 < K2( ˜p2; B1) in Table II can be directly
justified
K2( ˜p2; A1) = [−0.0745232, −0.0579265] < 0 K2( ˜p2; B1) = [0.0785704, 0.1848471] > 0.
The parameters chosen here also satisfy the convergence theorem that is not included in this paper. It can be observed in Fig. 4 that almost all trajectories converge to three equilibria marked in red (the other two equilibria are marked in green). These five equilibria can be computed numerically as
(1.5582, 1.9426), (−0.15341, 1.691), (−1.2876, 1.5082), (−1.6522, −1.0216), (−1.5578, −0.30005).
Example 2: This example illustrates Theorems 2 and 3
for (1) with n= 3. We consider the parameters (μi) = ⎛ ⎝11 1 ⎞ ⎠ , (αi j) = ⎛ ⎝01.05.8 01.05.9 00 0 0.05 0.6 ⎞ ⎠ (Ii) = ⎛ ⎝0.050 0.15 ⎞ ⎠ , (βi j) = ⎛ ⎝ 00.2 00.1 0.050.05 0.05 0 0.1 ⎞ ⎠ . It is straightforward to see i = 1, 2 ∈ B33 and i = 3 ∈ M, and thus card(B33) = 2. In addition, we set τii = 0.1, τi j = 12 for i, j = 1, 2, 3, i = j. We compute to find ˜p1 = ˜p2 =
−0.8813736, ˜q1= ˜q2= 0.8813736, k1+= 0.15, k1−= −0.05, k2+= 0.1, k−2 = −0.1, k3+= 0.25, and k−3 = 0.05. First, let us
examine the conditions in Theorem 2. For (7), for i = 1 ∈ B33, we take θ1= 0.3 ∈ (0, 1); then
θ1= 0.3 > ¯L2|α12+ β12| + ¯L3|α13+ β13| = 0.107.
For i= 2 ∈ B33, we take θ2= 0.4 ∈ (0, 1); then
θ2= 0.4 > ¯L1|α21+ β21| + ¯L3|α23+ β23| = 0.107.
For i= 3 ∈ M, we take θ3= 0.2 ∈ (0, 1); then
θ3= 0.2 > ¯L1|α31+ β31| + ¯L2|α32+ β32| = 0.1.
For (8), for i = 1 ∈ B3
3, we takeθ1 = 0.3 ∈ (0, 1); then for
ξ ∈ (−∞, ˆa1] ∪ [ˇc1, ∞) max g1(ξ) = 0.25098 < 0.35 = μ1− θ1 α11+ β11 and for ξ ∈ [ ˆb1, ˇb1] min g1(ξ) = 0.97723 > 0.65 = μ1+ θ1 α11+ β11.
For i= 2 ∈ B33, we take θ2= 0.4 ∈ (0, 1); then
max g2(ξ) = 0.23747 < 0.3 =
μ2− θ2
α22+ β22
forξ ∈ (−∞, ˆa2] ∪ [ˇc2, ∞), and
min g2(ξ) = 0.98996 > 0.7 =
μ2+ θ2
α22+ β22
forξ ∈ [ ˆb2, ˇb2]. For i = 3 ∈ M, we take θ3 = 0.2 ∈ (0, 1);
then forξ ∈ [ ˇm3, ˆm3]
max g3(ξ) = 0.97402 < 1.14286 = μ3− θ3 α33+ β33.
Next, let us examine the conditions in Theorem 3. For (22), for i= 1 ∈ B33, we have ˆF1( ˜p1) = −0.2188399 < 0, ˇF1( ˜q1) = 0.3188399 > 0. For i= 2 ∈ B33, we have ˆF2( ˜p2) = −0.3508400 < 0, ˇF2( ˜q2) = 0.3508400 > 0. For (25) β11τ11 = 0.02 < τ11c = 0.2439024 β22τ22 = 0.01 < τ11c = 0.2439024 β33τ33 = 0.01 < τ11c = 0.2333333.
We need the following quantities to examine (26): ˆaF 1 = −1.4941831, ˆb F 1 = −0.3387856, ˆc F 1 = 2.2719172 ˇaF 1 = −2.1616717, ˇb1F= 0.2210641, ˇc1F = 1.6413113 ˆaF 2 = −1.6850199, ˆb F 2 = −0.1862482, ˆc F 2 = 2.1258384 ˇaF 2 = −2.1258384, ˇb2F= 0.1862482, ˇc2F = 1.6850199 ˆmF 3 = 0.7054875, ˇm F 3 = 0.0664389.
For (26), for i = 1 ∈ B33, we takeθ1= 0.3 ∈ (0, 1); then for
ξ ∈ (−∞, ˆaF 1] ∪ [ˇc1F, ∞) max g1(ξ) = 0.30935 < 0.35 = μ1− θ1 α11+ β11 and for ξ ∈ [ ˆb1F, ˇb1F] min g1(ξ) = 0.89704 > 0.65 = μ1+ θ1 α11+ β11.
