類神經網路的數學研究(II)

行政院國家科學委員會專題研究計畫 期中進度報告

類神經網路的數學研究(2/3)

中 華 民 國 94 年 5 月 25 日

離散時間類神經網路的全局一致性

我們研究離散時間型的 Grossberg’s Model 之全局一致性；亦即討

論在什麼條件下，任一起始點皆會趨近於一平衡狀態。一般而言，此

類問題的研究皆採用 LaSalle’s invariance principles，如果可以找到適

合的 Lyapunov function，而在這個模型中，並不知道存在有任何

Lyapunov function。我們運用數學分析，研究解的各 component 的

Global Consensus for Discrete-time

Neural Networks

Jui-Pin Tseng and Chih-Wen Shih*

∗

Department of Applied Mathematics

National Chiao Tung University

Hsinchu, Taiwan, R.O.C.

May 25, 2005

Abstract

A discrete-time version of Grossberg’s competitive system is studied. We are interested in how the dynamics of the system reach global consensus. Analytical arguments are developed to conclude that every orbit converges to a fixed point point of the system as time tends to infinity, without knowing a Lyapunov function.

1

Introduction

Global consensus (also named convergence of dynamics, or complete stability) has been an important dynamical behavior in the theory and applications of competitive systems and neural network systems. The commonest ways to guarantee convergence of dynamics is to find a Lyapunov function for the system, that is, a continuous real valued function V on state space, which is nonincreasing along trajectories of the system. One then applies the LaSalle’s invariance principle to conclude the convergence. For example, Cohen and Grossberg (1983) [1] proved one convergence theorem for neural network systems of the form

˙ xi = ai(x)[bi(xi) − n X j=1 ωijgj(xj)], i = 1, · · · , n, (1.1)

∗_{Author for Correspondence. This work is partially supported by The National Science Council,}

where ai ≥ 0, the matrix [ωij] of coupling weights is symmetric, and g

0

j ≥ 0 for all

j. There exists a Lyapunov function V (x) = − n X i=1 Z _xi 0 bi(ξ)g 0 i(ξ)gξ + 1 2 n X i=1 n X j=1 ωijgi(xi)gj(xj).

They showed that if ai > 0 and g

0

i > 0 for every i, then V is a strict Liapunov

function and therefore the system is quasi-convergent, see also [9]. Forti et. al. (1995) [2] proved global stability of Hopfield-type neural network of the form

˙ xi = −dixi+ n X j=1 Tijgj(xj) + Ii, (1.2)

where di > 0, gj is nondecreasing function. Again, the results obtained therein

employed a Lyapunov function of the so-called generalized Lur’e-Postnikov type. However, it is not always easy to find a suitable Lyapunov function when considering convergent dynamics. Grossberg (1978) [3] proved a convergence theorem for a class of “competitive systems” for which no Lyapunov functions are known. He considered systems of the form

˙

xi = ai(x)[bi(xi) − c(x)], (1.3)

where ai ≥ 0, _∂x∂c_i ≥ 0, for i = 1, · · · , n. Herein, each bi is a function of only

one variable xi, and the function c does not depend on i. In this kind of system,

population xi at neuron i competes indirectly with other xj through a scalar c(x),

i.e., the interaction among neurons are through function c(x). Worth noticed, it is difficult to find a suitable Lyapunov function for (1.3). In fact, systems (1.1), (1.2) both can be written in the form

˙

xi = ai(x)[bi(xi) − ci(x)],

which has a crucial difference from (1.3).

The “competition” for (1.3) by Grossberg means ai ≥ 0,_∂x∂c_j ≥ 0, for all i, j

and therefore has a little different sense from the commonly used one. Usually , a

system ˙xi = Gi(x1, x2, · · · , xn) is competitive if ∂G_∂xji ≤ 0, for i 6= j. The sense of

competition in Grossberg’s paper can be seen if we consider functions ai as positive

constants. The assumption on ai for the studied dynamics is more general though.

Let us give more details to Grossberg’s model. In (1.3), n is any integer

any number of competing populations, any interpopulation signal functions bi(xi),

any mean competition function, or adaptation level c(x), and any state-dependent

amplifications ai(x) of the competitive balance. That work in [3] proved that any

initial value x(0) ≥ 0 (i.e. xi(0) ≥ 0, for any i) generates a limiting pattern x(∞) =

(x1(∞), x2(∞), · · · , xn(∞)) with 0 ≤ xi(∞) := limt→∞xi(t) < ∞, under some

conditions on ai, bi, c. We shall summarize the main ideas of Grossberg’s work in

Section 4.

Recently, discrete-time systems have attracted much scientific interests, cf. [5], [6], [8]. In this study, we consider the following discrete-time version of Grossberg’s model

xi(k + 1) = xi(k) + βai(x(k))[bi(xi(k)) − c(x(k))], (1.4)

where i = 1, 2, · · · , n, k ∈ N0 := {0}

S

N. Viewing from the δ-operator, (1.3) can be approximated by

xi((k + 1)δ) = xi(kδ) + δai(x(kδ))[bi(xi(kδ)) − c(x(kδ))]. (1.5)

One usually takes xi[k]δ := xi(kδ) as the k-th iteration of xi and

x[k]δ := (x1[k]δ, x2[k]δ, · · · , xn[k]δ) the k-th iteration of x.

In this presentation, we mainly consider (1.4) with β = 1, i.e.

xi(k + 1) = xi(k) + ai(x(k))[bi(xi(k)) − c(x(k))]. (1.6)

We define ∆xi(k) := xi(k + 1) − xi(k), hence system (1.6) can be rewritten in the

form

∆xi(k) = ai(x(k))[bi(xi(k) − c(x(k))]. (1.7)

The main purpose of this investigation is to find out under what conditions on

functions ai, bi, and c, systems (1.4) or (1.6) possesses a global limiting pattern

x(∞) := (x1(∞), x2(∞), · · · , xn(∞)) with −∞ < xi(∞) := limt→∞xi(t) < ∞ for

every i, given any initial value x(0).

Below, in Section 2, we state the main results of this presentation. In Section 3, we prove three key lemmas for our main result Theorem 2.2. In Section 4, we summarize the work of Grossberg [3] and make a generalization. A comparison of the analysis in deriving the global consensus for the continuous-time and the discrete-time competitive systems is also made.

2

Main Results

Definition 2.1. (Global Consensus) A competitive system is said to achieve

global consensus (or global pattern information) if, given any initial value x(0), the limit xi(∞) := limk→∞xi(k) exist, for all i = 1, 2, · · · , n.

The main results require the following conditions :

Condition (I): Each ai(x) is continuous, and

0 < ai(x) ≤ 1, for all x ∈ Rn, i = 1, · · · , n. (2.1)

Condition (I)0_{: Each a}

i(x) is continuous, and

0 < ai(x) ≤ A, for all x ∈ Rn, i = 1, · · · , n. (2.2)

Condition (II): c(x) is bounded and continuously differentiable with bounded

deriva-tives; namely, there exist constants M1, M2, rj such that

M1 ≤ c(x) ≤ M2, (2.3)

0 ≤ ∂c

∂xj

(x) ≤ rj, (2.4)

for all x ∈ Rn_{, and j = 1, 2, · · · , n.}

Condition (III): bi(ξ) is continuously differentiable, strictly decreasing and there

exist di > 0, li ∈ R, ui ∈ R such that

−di ≤ b0i(ξ) < 0, for all ξ ∈ R, (2.5)

and

bi(ξ) > M2, for ξ ≤ li, (2.6)

bi(ξ) < M1, for ξ ≥ ui. (2.7)

Condition (IV): For i = 1, · · · , n,

0 < di ≤ 1 −

n

X

j=1

rj < 1. (2.8)

Condition (IV)0_{: For i = 1, · · · , n,}

0 < di ≤ 1 β − n X j=1 rj < 1 β. (2.9)

Condition (IV)00_{: For i = 1, · · · , n,} 0 < di ≤ 1 Aβ − n X j=1 rj < 1 Aβ. (2.10) Set d := min{di : i = 1, 2, · · · , n}, (2.11) M := max{|M1|, |M2|}. (2.12)

Theorem 1. System (1.6) with functions ai,bi,and c satisfying Conditions (I), (II),

(III), and (IV) achieves global consensus.

The proof of Theorem 2.2 consists of three lemmas stated below. For system (1.4), the following corollary can be derived.

Corollary 2. System (1.4) with functions ai, bi, and c satisfying Conditions (I),(II),

(III), and (IV)0 _{achieves global consensus.}

In fact, we only need that function ai is continuous, positive and bounded

above by some real number, say A, for all i, instead of Condition (I). It is due to that (1.6) can be rewritten as

xi(k + 1) = xi(k) +

ai(x(k))

A [Abi(xi(k)) − Ac(x(k))].

We thus derive the following Corollary.

Corollary 3. System (1.4) whose functions ai, bi,and c satisfy Condition (I)0, (II),

(III), and (IV)00 _{achieves global consensus.}

Remark 2.1. From Corollary 2, we find that the smaller β in (1.4) (δ in (1.5)) is,

the weaker restrictions on functions a1, bi, c are. In other words, when we consider

(1.4) in stead of (1.3), and want to have the global consensus proposition, we must

choose sufficiently small β in (1.4), basically.

In order to state the key lemmas for our main result, Theorem 2.2, we introduce some notations and definition as follows:

Notation 2.2.

gi(k) = bi(xi(k) − c(x(k)),

∆gi(k) = gi(k + 1) − gi(k),

gmax(k) = max{gi(k) : i = 1, 2, · · · , n},

gmin(k) = min{gi(k) : i = 1, 2, · · · , n},

I(k) = min{i : gi(k) = gmax(k)}, J(k) = min{i : gi(k) = gmin(k)}, ˆ x(k) = xI(k)(k), ˇ x(k) = xJ(k)(k), ˆb(k) = bI(k)(ˆx(k)), ˇb(k) = bJ(k)(ˇx(k)), ∆ˆb(k) = ˆb(k + 1) − ˆb(k), ∆ˇb(k) = ˇb(k + 1) − ˇb(k), ∆bi(xi(k)) = bi(xi(k + 1)) − bi(xi(k)).

Definition 2.3. (i) A jump of type-1 is said to occur from i to j at kth iteration if

I(k) = i, I(k+1)=j, (ii) A jump of type-2 is said to occur from i to j at kth iteration if J(k) = i, J(k + 1) = j.

Lemma 1. Consider system (1.6) with ai, bi, and c satisfying (2.1), (2.3), (2.5),

(2.6) and (2.7). Given any initial value x(0), {x(k)} will be attracted to some

compact cube contained in Rn_{. Hence sequence {x}

i(k) | k ∈ N0} are bounded above

and below for all i = 1, 2, · · · , n.

If Lemma 1 is valid, consider an arbitrary orbit {x(k)}. Then {|ai(x(k))| |

k ∈ N0} is bounded below by some positive number, say 0 < ρi ≤ |ai(x(k))| for all

k ∈ N0 and {b

0

i(xi(k)) | k ∈ N0} are bounded above by some negative number, say

b0_i(xi(k)) ≤ −²i < 0 for all k ∈ N0. We define

ρ := min{ρi : i = 1, 2, · · · , n}, (2.13)

² := min{²i : i = 1, 2, · · · , n}. (2.14)

Lemma 2. Consider system (1.6) with ai, bi, and c satisfying (2.1), (2.4), (2.5)

(I) for function gmax, either case (gmax-(i)) or case (gmax-(ii)) holds, where

(gmax-(i)): gmax(k) < 0, for all k ∈ N0,

(gmax-(ii)): gmax(k) ≥ 0, for all k ≥ k1, for some k1 ∈ N0;

(II) for function gmin, either case (gmin-(i)) or case (gmin-(ii)) holds, where

(gmin-(i)): gmin(k) > 0, for all k ∈ N0,

(gmin-(ii)): gmin(k) ≤ 0, for all k ≥ k2, for some k2 ∈ N0.

If Lemma 2 holds, there are only four possibilities to consider.

case (i): Both (gmax-(i)) and (gmin-(i)) hold. This case is impossible from our

definition of gmax and gmin.

case (ii): Both (gmax-(i)) and (gmin-(ii)) hold, then sequence {xi(k)} will always

decrease as k increases, for each i = 1, 2, · · · , n. By Lemma 1, {xi(k)} are bounded

below for everyi, hence the limit xi(∞) exists, for i = 1, 2, · · · , n.

case (iii): Both (gmax-(ii)) and (gmin-(i)) hold, then sequence {xi(k)} will always

increase as k increases, for all i = 1, 2, · · · , n. By Lemma 1, {xi(k)} are bounded

above for every i. Hence the limit xi(∞) exists, for every i = 1, 2, · · · , n.

case (iv): Both (gmax-(ii)) and (gmin-(ii)) hold.

Accordingly, we are left with the case case (iv) only, for the conclusion of

global consensus for (1.6). We thus assume that gmax(0) ≥ 0, gmin(0) ≤ 0, without

loss of generality.

Lemma 3. Consider system (1.6) with ai, bi, and c satisfying Conditions (I), (II),

(III), and (IV) then,

(i) limk→∞ˆb(k) exists, denoted by ˆB, and limk→∞c(x(k)) = ˆB,

(ii) limk→∞ˇb(k) exists, denoted by ˇB, and limk→∞c(x(k)) = ˇB.

If Lemma 3 holds, we find that lim

k→∞ˆb(k) = limk→∞ˇb(k) =: ¯B, (2.15)

since limk→∞c(x(k))= ˆB= ˇB. For any i = 1, 2, · · · , n, gmin(k) ≤ gi(k) ≤ gmax(k), for

all k ∈ N0. Equivalently,

ˇb(k) − c(x(k)) ≤ bi(xi(k)) − c(x(k)) ≤ ˆb(k) − c(x(k)),

for all k ∈ N0. Thus, ˇb(k) ≤ bi(xi(k)) ≤ ˆb(k), for all k ∈ N0. Therefore

lim

We obtain

lim

k→∞ˆb(k) = limk→∞bi(xi(k)) = limk→∞ˇb(k) = ¯B,

by (2.15). Therefore we conclude that lim

k→∞bi(xi(k)) = ¯B, for all i = 1, 2, · · · , n. (2.16)

Moreover, limk→∞xi(k) exists, for every i = 1, 2, · · · , n, by (2.5) and (2.16). Hence,

global consensus of system (1.6) is achieved, if functions ai, bi, and c satisfy

Condi-tions (I), (II), (III), (IV).

3

Proofs of Lemmas

Proof of Lemma 1 : For any initial vale x(0), we consider the iteration sequence

{xi(k)} and their components xi(k). We divide the proof into several steps.

(i) By (2.3) and(2.7),

bi(xi) − c(x) < 0, (3.1)

for all xi ≥ ui. Therefore

∆xi(k) = ai(x(k))[bi(xi(k) − c(x(k))] < 0, (3.2)

if xi(k) ≥ ui. Similarly, By (2.3) and (2.6),

bi(xi) − c(x) > 0, (3.3)

for all xi ≤ li. Therefore

∆xi(k) = ai(x(k))[bi(xi(k) − c(x(k))] > 0, (3.4)

if xi(k) ≤ li. We claim that for all k ∈ N0,

|bi(xi(k))| ≤ di|xi(k)| + |bi(0)|. (3.5)

This follows from

bi(xi(k)) − bi(0) = b

0

i(·)xi(k),

where “·” means some real number between xi(k) and 0. Thus, by (2.5),

|bi(xi(k))| = |bi(0) + b 0 i(·)xi(k)| ≤ |bi(0)| + |b 0 i(·)xi(k)| ≤ |bi(0)| + di|xi(k)|.

(ii) Next, we show that for fixed constant Li, there exist some constants u 0 i and d 0 i, where u0_i > 0, 0 < di < d 0 i < 1 such that di|xi| + Li < d 0 i|xi|, if |xi| ≥ u 0 i. (3.6)

Let us verify this. Notably,

di|xi| + Li

|xi|

= di+ Li

|xi|

→ di < 1,

as |xi| → ∞. Therefore, there exist some u

0 i and d 0 i, where u 0 i > 0, 0 < di < d 0 i < 1 such that (di|xi| + Li)/|xi| < d0i, if |xi| ≥ u 0 i. (iii) |∆xi(k)| = |ai(x(k))[bi(xi(k) − c(x(k))]| ≤ |bi(xi(k) − c(x(k))| (by (2.1)) ≤ |bi(xi(k))| + |c(x(k))| ≤ di|xi(k)| + |bi(0)| + |c(x(k))| (by (3.5)) ≤ di|xi(k)| + |bi(0)| + M (by (2.3), (2.12)).

Hence, by (3.6), we choose |bi(0)| + M = Li, there exist some constants u

0 i and d 0 i where u0_i > 0, 0 < di < d 0 i < 1 such that |∆xi(k)| < d 0 i|xi(k)| < |xi(k)|, if |xi(k)| ≥ u 0 i. (3.7)

(iv) Set, for each i,

q_i0 := max{|ui|, |li|, u

0

i}. (3.8)

Let Q0 _{:= [−q}0

1, q10] × · · · × [−q0n, qn0]. Q0 is a compact set, hence |ai(x)[bi(xi) − c(x)]|

is bounded on Q0_{, say}

|ai(x)[bi(xi) − c(x)]| ≤ K, (3.9)

for all x ∈ Q0, for all i. Set

qi := q

0

i+ K, (3.10)

Q := [−q1, q1] × · · · × [−qn, qn]. (3.11)

We shall utilize (3.2), (3.4), (3.7), (3.8), (3.9), (3.10) in the following discus-sions.

case (a): If xi(0) ∈ [−qi, −q 0 i], then ∆xi(0) > 0, due to xi(0) ≤ −q 0 i ≤ li, and |∆xi(0)| < |xi(0)|, due to xi(0) ≤ −u 0

i, hence xi(1) still stays in (−qi, −q

0

i], or moves

into (−q_i0, q_i0). If the former case occurs, we consider xi(1) as case (a) again. If the

latter case occurs, we consider xi(1) as in the following case (b).

case (b): If xi(0) ∈ (−q

0

i, q

0

i), then |∆xi(0)| < K, by (3.9), hence xi(1) will stay

in [−qi, −q 0 i] or (−q 0 i, q 0 i) or [q 0

i, qi]. Then we can still consider xi(1) as in case (a),

case (b), and case (c), respectively.

case (c): If xi(0) ∈ [q 0 i, qi], then ∆xi(0) < 0, by xi(0) ≥ q 0 i ≥ ui, and |∆xi(0)| < |xi(0)|, by xi(0) ≥ u 0

i, hence xi(1) still stays in [q

0 i, qi), or moves into (−q 0 i, q 0 i). If

the former case occurs, we consider xi(1) as in case (c) again. If the latter case

occurs, we consider xi(1) as in case (b). From the above arguments, we find that if

−qi ≤ xi(0) ≤ qi, then −qi < xi(1) < qi, and we can prove that −qi < xi(k) < qi,

for all k ≥ 2, by induction.

(vi): If xi(0) < −qi, then

case (d): {xi(k)} either increases as k increases and remains bounded above

by −qi, or

case (e): {xi(k)} enter [−qi, qi] at some iteration, and never leaves [−qi, qi]

again.

(vii) if xi(0) > qi, then

case (f): {xi(k)} either decreases as k increases and remains bounded below

by qi, or

case (g): {xi(k)} enters [−qi, qi] at some iteration, and never leaves [−qi, qi]

again.

We find that no matter which case above occurs, {xi(k)} are bounded above

and below for all i. Therefore, {|ai(x(k))|} are bounded below by some positive

number, say 0 < ρi ≤ |ai(x(k))|, and {b

0

i(xi(k))} are bounded above by some

nega-tive number, say b0_i(xi(k)) ≤ −²i < 0. In fact, it is impossible for the above case (d)

and case (f) to occur. This is due to that if case (d) occurs, then

bi(xi(k)) − c(x(k)) = bi(xi(k)) − bi(li) + bi(li) − c(x(k))

> bi(xi(k)) − bi(li)

= b0_i(·)[xi(k) − li]

≥ ²iK,

for all xi(k) ≤ −qi ≤ li − K, by (2.5), (3.3), where “·” means some real number

Hence {xi(k)} will increase unboundedly, and this yields a contradiction. Therefore

case (d) never occurs. Similarly, case (f) never occurs too. By the arguments above, we can find that given any initial value x(0), {x(k)} will be attracted by Q.

Proof of Lemma 2:

For function gmax, if gmax(k) ≥ 0 for some k, say I(k) = i, then gj(k) ≤ gi(k),

for all j 6= i. Consider two possibilities |∆gi(k)| ≤ gi(k), and |∆gi(k)| > gi(k).

case (i) |∆gi(k)| ≤ gi(k): It follows that

gmax(k + 1) ≥ gi(k + 1) = gi(k) + ∆gi(k) ≥ 0.

case (ii) |∆gi(k)| > gi(k). Let us elaborate.

∆gi(k) = gi(k + 1) − gi(k) = bi(xi(k + 1)) − c(x(k + 1)) − [bi(xi(k)) − c(x(k))] = bi(xi(k + 1)) − bi(xi(k)) − [c(x(k + 1)) − c(x(k))] = b0_i(·)[xi(k + 1) − xi(k)] − n X j=1 ∂c ∂xj (•)[xj(k + 1) − xj(k)],

where “·” means some real number between xi(k + 1) and xi(k), “•” means some

vector between x(k + 1) and x(k). Thus,

∆gi(k) = b 0 i(·)ai(x(k))gi(k) − n X j=1 ∂c ∂xj (•)aj(x(k))gj(k) ≥ −diai(x(k))gi(k) − n X j=1 rjaj(x(k))gi(k) (by (2.3), (2.5) and gj(k) ≤ gi(k) ≥ 0) ≥ −digi(k) − n X j=1 rjgi(k) (by (2.1)) = (−di− n X j=1 rj)gi(k) ≥ −gi(k) (by (2.8)).

Hence ∆gi(k) > 0, since |∆gi(k)| > gi(k) and ∆gi(k) ≥ −gi(k). Therefore, gmax(k +

1) ≥ gi(k + 1) = gi(k) + ∆gi(k) > 0.

For function gmin, if gmin(k) ≤ 0 for some k, say J(k) = i. Then gj(k) ≥ gi(k),

case (i) |∆gi(k)| ≤ −gi(k): It follow sthat gmin(k+1) ≤ gi(k+1) = gi(k)+∆gi(k) ≤ 0. case(ii) |∆gi(k)| > −gi(k): ∆gi(k) = gi(k + 1) − gi(k) = bi(xi(k + 1)) − c(x(k + 1)) − [bi(xi(k)) − c(x(k))] = bi(xi(k + 1)) − bi(xi(k)) − [c(x(k + 1)) − c(x(k))] = b0_i(·)[xi(k + 1) − xi(k)] − n X j=1 ∂c ∂xj (•)[xj(k + 1) − xj(k)],

where “·” means some real number between xi(k + 1) and xi(k), “•” means some

vector between x(k + 1) and x(k). Thus

|∆gi(k)| = b 0 i(·)ai(x(k))gi(k) − n X j=1 ∂c ∂xj (•)aj(x(k))gj(k) ≤ −diai(x(k))gi(k) − n X j=1 rjaj(x(k))gi(k) (by (2.3), (2.5) and gj(k) ≥ gi(k) ≤ 0) ≤ −digi(k) − n X j=1 rjgi(k) (by (2.1)) = (di+ n X j=1 rj)(−gi(k)) ≤ −gi(k) (by (2.8)).

Hence ∆gi(k) < 0, since |∆gi(k)| > −gi(k) and ∆gi(k) ≤ −gi(k). So, gmin(k + 1) ≤

gi(k + 1) = gi(k) + ∆gi(k) < 0.

From the above arguments, we find that function gmaxmay keep negative at all

iterations. But once it becomes nonnegative at some iteration, it will always remain

nonnegative after this iteration. Similarly, gmin may keep positive at all iterations.

But once it get nonpositive at some iteration, it will always be nonpositive after this iteration. This completes the proof of Lemma 2. With Lemma 2, we assume that

gmax(0) ≥ 0, gmin(0) ≤ 0, without loss of generality.

Proof for Lemma 3:

We assert that limk→∞ˆb(k) exists, and denote it by ˆB; moreover, limk→∞c(x(k))= ˆB.

Case (i): There exist finitely many jumps of type-1.

In this case, there exist some K3 ∈ N, some i, say 1, such that gmax(k) = g1(k) ≥ 0,

{x1(k)} are bounded above. Therefore, limk→∞x1(k) exists, hence limk→∞b1(x1(k))

exists, denoted by ˆB. Restated, limk→∞ˆb(k) = ˆB.

Next, we justify that limk→∞c(x(k))= ˆB. Assume otherwise, limk→∞c(x(k)) 6=

ˆ

B. It follows from gmax(k) = g1(k) ≥ 0, for all k ≥ K3, that b1(x1(k)) ≥ c(x(k)),

for all k ≥ K3. There exists some ε > 0, and subsequence {kl}∞l=1 of positive

integer number with k1 > K3 such that |c(x(kl)) − ˆB| > ε, for all l ∈ N. Because

limk→∞b1(x1(k)) = ˆB, for such ε, there exists K4 ∈ N, such that |b1(x1(k))− ˆB| ≤ ε₂,

for all k ≥ K4. Therefore g1(kl) = b1(x1(kl)) − c(x(kl)) > ε₂, for all kl ≥ K4. We

find that {x1(k)} is always increasing after K4-th iteration. In fact,

∆x1(kl) = a1(x(kl))[b1(x1(kl) − c(x(kl))] > ρ

ε

2,

if kl ≥ K4. Hence {x1(k)} will increase unboundedly, and yields a contradiction to

Lemma 1.

Case (ii): There exist infinitely many jumps of type-1.

We shall justify that {ˆb(k)} decreases as {k} ↑ ∞. Consider a fixed k ∈ N. Subcase (ii-a): no jump of type-1 occurs at k-th iteration.

Suppose I(k) = I(k + 1) = i, then gi(k) ≥ 0, gi(k + 1) ≥ 0. In addition,

ˆb(k + 1) = bi(xi(k + 1))

≤ bi(xi(k))

= ˆb(k),

thank to (2.5), and ∆xi(k) = ai(x(k))gi(k) ≥ 0. Thus {ˆb(k)} decreases as k

in-creases.

Subcase (ii-b): jump of type-1 occurs at k-th iteration and gi(k) ≥ 0, gj(k) ≥ 0,

where I(k) = i 6= I(k + 1) = j. It follows that

ˆb(k + 1) = bj(xj(k + 1))

≤ bj(xj(k))

≤ bi(xi(k))

= ˆb(k),

due to (2.5), ∆xj((k)) = aj(x(k))gj(k) ≥ 0, and by I(k) = i 6= j.

Subcase (ii-c): jump of type-1 occurs at k-th iteration and gi(k) ≥ 0, gj(k) < 0,

Notably, we still have gj(k + 1) ≥ 0. We claim that bj(xj(k + 1)) − bj(xj(k)) ≤ bi(xi(k)) − bj(xj(k)). (3.12) Indeed, LHS = b0_j(·)∆xj(k) = b0_j(·)aj(x(k))gj(k) ≤ b0_j(·)gj(k) (by (2.1)) ≤ −digj(k) (by (2.5), and gj(k) < 0)) ≤ gi(k) − gj(k) (by (1 − di)gj(k) < 0 ≤ gi(k)) = bi(xi(k)) − bj(xj(k)) = RHS.

Herein, “ · ” is defined as before. Hence, ˆb(k + 1) = bj(xj(k + 1)) ≤ bi(xi(k)) = ˆb(k).

All these cases indicate that ˆb(k) decreases as k increases. By Lemma 1, {x(k)} are

attracted into some compact cube Q contained in Rn_{. Therefore, {b}

i(xi(k))} are

bounded below, and so are {ˆb(k)}. Hence {ˆb(k)} decreases and converges to some

number ˆB as k tends to infinity (denoted by {ˆb(k)} ↓ ˆB).

Next, we verify that limn→∞c(x(k))= ˆB. Assume otherwise: limk→∞c(x(k)) 6=

ˆ

B. There exist some positive µ, subsequence {kl}∞l=1 of positive integers, such that

|c(x(kl)) − ˆB| >

µ

²ρ, (3.13)

Where ², ρ are defined in (2.13) and (2.14). Because {ˆb(k)} ↓ ˆB, for µ0 _{:= min{}µ

²ρ, µ} >

0, there exists L ∈ N such that ˆ

B ≤ bI(k)(xI(k)(k)) ≤ ˆB + µ0, (3.14)

for all k ≥ L. Moreover

gmax(`) = bI(`)(xI(`)(`) − c(x(`)) ≥ 0, (3.15)

for all ` ∈ N. Let kL > L. By (3.13), (3.14), and (3.15), we have

gmax(kL) = b1(x1(kL)) − c(x(kL)) >

µ ²ρ,

where, for convenience, we set I(kL)=1 without loss of generality. There are two

possibilities at kL-th iteration, either jump of type-1 occurs or not. If it does not

occur, then |∆ˆb(kL)| = |ˆb(kL+ 1) − ˆb(kL)| = |b1(x1(kL+ 1)) − b1(x1(kL))| = |b0₁(·)||x1(kL+ 1) − x1(kL)| = |b0₁(·)||a1(x(kL))||g1(kL)| = |b0₁(·)||a1(x(kL))||gmax(kL)| > ²ρµ ²ρ = µ. But it is impossible, because of (3.14).

If jump of type-1 occurs at kL-th iteration. Assume that I(kL+ 1)=2. Below

we consider three different cases for b2(x2(kL)):

Case (a): ˆB ≤ b2(x2(kL)) < b1(x1(kL)). Then g2(kL) > _²ρµ, and |∆b2(x2(kL))| =

|b0₂(·)||a2(x(kL))||g2(kL)| > ²ρ_²ρµ = µ. It is impossible, due to (3.14).

Case (b): ˆB > b2(x2(kL)) ≥ c(x(kL)). Then g2(kL) ≥ 0, and x2(kL+ 1) ≥

x2(kL). Thus,

ˆb(kL+ 1) = b2(x2(kL+ 1))

≤ b2(x2(kL))

< B.ˆ

It is impossible, since {ˆb(k)} ↓ ˆB.

Case (c): b2(x2(kL)) < c(x(kL)). Then g2(kL) < 0, and

∆b2(x2(kL)) = b2(x2(kL+ 1)) − b2(x2(kL))

= b0₂(·)a2(x(kL))g2(kL)

≤ −d2g2(kL)

≤ −g2(kL).

Thus, b2(x2(kL+1)) = b2(x2(kL)) + ∆b2(x2(kL)) < b2(x2(kL)) − g2(kL) = c(x(kL).

Hence ˆb(kL+ 1) = b2(x2(kL+ 1)) < c(x(kL)) < ˆB. It is impossible, since {bmax(k)} ↓

ˆ

From the above discussions, we conclude that limk→∞c(x(k)) = ˆB.

The second part of the lemma asserts that limk→∞ˇb(k) exists, denoted by ˇB,

and limk→∞c(x(k))= ˇB. The proof for the assertion resembles the first part. Let us

elaborate.

Case (i): If there exist finitely many jumps of type-2, there exists some K5 ∈ N

, some i, say 1, such that gmin(k) = g1(k) ≤ 0, for all k ≥ K5, then {x1(k)}

will be decreasing as k ↑ ∞. By Lemma 1, {x1(k)} are bounded below,

there-fore limk→∞x1(k) exists, hence limk→∞b1(x1(k)) exists, denoted by ˆb. (Namely,

limk→∞bmin(k) = ˆb.)

claim : limk→∞c(x(k))=ˆb.

Because gmin(k) = g1(k) ≤ 0, for all k ≥ K5, b1(x1(k)) ≤ c(x(k)), for all k ≥ K5.

Assume otherwise: limk→∞c(x(k)) 6= ˆb.

There exists some ε > 0, and subsequence {kj}∞j=1 of positive integer number with

k1 > K5 such that |c(x(kj)) − ˆb| > ε , for all j ∈ N. Because lim`→∞b1(x1(`)) = ˆb,

for such ε , there exists K6 ∈ N, such that |b1(x1(`)) − ˆb| ≤ ε<sub>2</sub>, for all ` ≥ K6.

Therefore g1(kj) = b1(x1(kj)) − c(x(kj)) < −ε₂, for all kj ≥ K6. We find that

{x1(k)} are always decreasing after K6− th iteration, especially at kj− th iteration,

∆x1(kj) = a1(x(kj))[b1(x1(kj) − c(x(kj))] < −ρε₂. Hence, {x1(k)} will decrease

un-boundedly, and make contradiction to Lemma 1. case (ii): If there exist infinitely many jumps of type-2.

claim : {bmin(k)} increase as {k} ↑ ∞.

For any k,

subcase (i): no jump of type-2 occurs at k − th iteration.

Say J(k) = J(k + 1) = i, therefore gi(k) ≤ 0, gi(k + 1) ≤ 0. bmin(k + 1) =

bi(xi(k + 1)) ≥ bi(xi(k)) (by (2.5), and ∆xi((k)) = ai(x(k))gi(k) ≤ 0) = bmin(k).

subcase (ii): jump of type-2 occurs at k − th iteration. Say J(k) = i 6= J(k + 1) = j, subcase (ii-a): gi(k) ≤ 0, gj(k) ≤ 0, bmin(k + 1) = bj(xj(k + 1)) ≥ bj(xj(k)) (by (2.5), ∆xj((k)) = aj(x(k))gj(k) ≤ 0) > bi(xi(k)) (by J(n) = i 6= j) = bmin(k) subcase (ii-b): gi(k) ≤ 0, gj(k) > 0,

At first, we check bj(xj(k + 1)) − bj(xj(k)) ≥ bi(xi(k)) − bj(xj(k)). (3.16) LHS = b0_j(·)∆xj(k) = b0_j(·)aj(x(k))gj(k) ≥ b0_j(·)gj(k) (by (2.1)) ≥ −djgj(k) (by (2.5), and gj(k) > 0)) ≥ gi(k) − gj(k) (by (1 − dj)gj(k) > 0 ≥ gi(k)) = bi(xi(k)) − bj(xj(k))

= RHS (where “ · ” defined as before). Hence,

bmin(k + 1) = bj(xj(k + 1))

≥ bi(xi(k)) (by (3.16))

= bmin(k)

After considering all cases, we can find that {bmin(k)} increase forever as {k}

↑ ∞. By Lemma 1, {x(k)} are attracted into some compact cube contained in

Rn_{, therefore {b}

i(xi(k))} are bounded above, and so are {bmin(k)}. Hence {bmin(k)}

increase and converge to some number, say ˆb (Denoted by {bmin(k)} ↑ ˆb).

claim : limk→∞c(x(k))=ˆb.

Assume otherwise: limk→∞c(x(k)) 6= ˆb.

There exist some positive ε, subsequence {kj}∞j=1 of positive integer number such

that

|c(x(kj)) − ˆb| >

ε

²ρ. (3.17)

Because {bmin(k)} ↑ ˆb, for ε

0

:= min{ε

²ρ, ε} > 0, there exists L ∈ N, such that

ˆb ≥ bJ(kj)(xJ(kj)(kj)) ≥ ˆb − ε

0

, for all j ≥ L. (3.18) Moreover

gmin(`) = bJ(`)(xJ(`)(`) − c(x(`)) ≤ 0, for all ` ∈ N. (3.19)

By (3.17), (3.18),and (3.19)

loss of generality).

At kL− th iteration, either jump of type-2 occurs or not

In latter case,

|∆bmin(kL)| = |bmin(kL+ 1) − bmin(kL)|

= |b1(x1(kL+ 1)) − b1(x1(kL))| = |b0₁(·)||x1(kL+ 1)) − (x1(kL))| = |b0₁(·)||a1(x(kL))||g1(kL)| = |b0₁(·)||a1(x(kL))||gmin(kL)| > ²ρε ²ρ = ε But it is impossible, since (3.18).

In former case, we say J(kL+ 1)=2.

Below we consider three different cases for b2(x2(kL)) as follows:

case (a): ˆb ≥ b2(x2(kL)) > b1(x1(kL)), then

g2(kL) < −_²ρε, |∆b2(x2(kL))| = |b 0 2(·)||a2(x(kL))||g2(kL)| > ²ρ_²ρε = ε. It is impossible, since (3.18). case (b): ˆb < b2(x2(kL)) ≤ c(x(kL)),then g2(kL) ≤ 0, x2(kL + 1) ≤ x2(kL), therefore bmin(kL + 1) = b2(x2(kL + 1)) ≥

b2(x2(kL)) > ˆb. It is impossible, since {bmin(k)} ↑ ˆb.

case (c): b2(x2(kL)) > c(x(kL)), then g2(kL) > 0, ∆b2(x2(kL)) = b2(x2(kL+ 1)) − b2(x2(kL)) = b 0 2(·)a2(x(kL))g2(kL) ≥ −d2g2(kL) ≥ −g2(kL). b2(x2(kL+1)) = b2(x2(kL)) + ∆b2(x2(kL)) > b2(x2(kL)) − g2(kL) = c(x(kL)). Hence bmin(kL + 1) = b2(x2(kL + 1)) > c(x(kL)) > ˆb. It is

impossible, since {bmin(k)} ↑ ˆb.

From above three cases, we can find that “ limk→∞c(x(k)) 6= ˆb” will make

contradiction, therefore limk→∞c(x(k)) = ˆb.

References

[1] Cohen, A. M. & Grossberg, S. Absolute Stability of Global Patttern

For-mation and Parallel Memory Storage by Competitive Neural Networks, IEEE

[2] Forti, M. & Tesi, A. New Conditions for Global Stability of Neural Networks

with Application to Linear and Quadratic Programming Problems, IEEE

Tran-sations on Circuit and Systems-I:Fundament Theory and Application, Vol. 42, No. 7(1995), pp.

[3] Grossberg, S. Competion, decision, consensus. J. Math. Anal. Appl. 66, 470(1978), pp..

[4] Grossberg, S. Preference order competition implies global limits in

n-dimention competition systems. 1977, preprint.

[5] H¨anggi, M., Reddy, H. C. & Moschytz, G. S. Unifying results in cnn

theory using delta operator, IEEE International Symposium on Circuits and

Sys-tems, 3(1999), pp. 547–550.

[6] Harrer, H. & Nossek, J. A. An analog implementation of discrete-time cnns, IEEE Transactions on Neural Networks, 3(1992), pp. 466–476.

[7] Hirsh, M. Convergent Activation Dynamics in Continuous Time Networks, Neural Netwok, Vol. 2 (1989), pp. 331-349.

[8] Sbitnev, V. I. & Chua, L. O. Local activity for discrete-map CNN Int. J. Bifur. Chaos, 12 (6) (2002), pp. 1227–1272.

[9] Shih, C.-W. & Weng, C.-W. Cycle-symmetric matrices and convergent neural