類神經網路的數學研究(I)

行政院國家科學委員會專題研究計畫 期中進度報告

類神經網路的數學研究(1/3)

中 華 民 國 93 年 5 月 25 日

中文摘要： Hopfied-type 類神經網路二十年來吸引很多科學研究者的興趣。基於其應用 上的目的與屬性，有時需要系統僅存在單一穩定平衡態。例如，在做組合最佳化 問題的計算時，希望系統中僅存在單一穩定平衡態，此即作為計算結果所提供之 最佳解。這一方面的理論發展的較為完備。 在我們的研究工作中，以幾何圖形的想法出發，建構出多平衡態解存在的條 件。經由估計線性化系統的特徵值，我們可以得到這些平衡解的穩定性的條件。 此外，我們引用了一個方法來估計這些穩定平衡解的 basin of attraction。因為推 導這條件的想法是有幾何的意涵，使得這些條件的檢驗都是很容易完成的，甚 且，很容易由電腦程式來檢驗。 這些常態解也同時是具延遲(delay)的類神經網路之常態解。有關 basins of attraction 之估計似乎也在這類的系統中有效，如果 delay 的大小有所限制的話。

Multiple Stationary Solutions and their Stability

for Hopfield-type Neural Networks

Chih-Wen Shih

Department of Applied Mathematics

National Chiao Tung University

Hsinchu, Taiwan, R.O.C.

*cwshih@math.nctu.edu.tw

May 25, 2004

Abstract

The number of stable stationary solutions corresponds to the memory ca-pacity for the neural networks. In this presentation, we address on multiple stable stationary solutions for Hopfield-type neural networks. Our goal is to formulate concrete parameter conditions to guarantee the existence of stable equilibria, through a suitable geometrical setting. We shall also derive con-ditions for the estimation of basins of attraction for these stable stationary solutions.

1

Introduction

Hopfield-type neural networks and their various generalizations have attracted much attention from the science community. The applications of these networks range from classifications, associative memory, to parallel computation and its ability in solving optimization problems. The theory on the dynamics of the networks has been developed according to the purposes of the applications. When a neural net-work is employed as an associative memory, the existence of many equilibria is a necessary feature. On the other hand, in the application to parallel computation and signal processing involving finding the solution of an optimization problem, the existence of a computable solution for all possible initial states is the best situation.

∗_{Author for Correspondence. This work is partially supported by The National Science Council,}

and The National Center of Theoretical Sciences, of R.O.C. on Taiwan. 1

Mathematically, this means that the network needs to have a unique equilibrium which is globally attractive. The theory on unique equilibrium for Hopfield-type neural networks is relatively well-known, cf . [3, 4]. In this presentation, we address on multiple stable equilibria for Hopfield-type neural networks.

In addition to the classical neural networks, we are also interested in the neural networks with delay, cf . [5]. The stationary equations are identical for the these systems with or without delays. Thus, confirmations for the existence of equilibrium points are valid for both systems. However, stability of the equilibrium points and dynamical behaviors can be very different for the systems with delay and without delay. It is very interesting to explore such a kind of difference as well as a possible coincidence of behaviors, say, as the delay is small. The comparisons for the dy-namical behaviors will also be performed via numerical simulations, to justify our theory as well as to reach the territory without theory.

We shall also derive conditions for the estimation of basins of attraction for these stable stationary solutions. A methodology for such a derivation has been developed in [2] for spatially discrete reaction-diffusion equations.

2

Existence of multiple equilibria and their

sta-bility

In this section, we shall formulate sufficient conditions for the existence of multiple equilibrium points. Since our approach is geometrical, these conditions are concrete and can be examined easily. We also derive stability criteria of these equilibria through estimations for the eigenvalues of the linearized system.

The classical Hopfield-type neural networks is given by dxi(t) dt = −bixi(t) + n X j=1 ωijgj(xj(t)) + Ji =: Fi(x(t)), i = 1, · · · , n. (2.1)

Recently, the Hopfield-type neural networks with delay has drawn much attention. A typical form for the neural network with delay is:

dxi(t) dt = −bixi(t) + n X j=1 ωijgj(xj(t − τij)) + Ji =: Fi(xt), i = 1, · · · , n, (2.2)

where 0 < τij ≤ r, and xt∈ C([−r, 0], Rn) is defined by xt(θ) = x(t + θ), θ ∈ [−r, 0].

all j,

gj(ξ) = g(ξ) :=

1

1 + e−ξ/ε, ε > 0.

For simplicity, we consider b = bi, ω = ωii for i = 1, · · · , n. Notably, the stationary

equation for systems (2.1) and (2.2) are identical. Namely,

−bixi+ n

X

j=1

ωijgj(xj) + Ji = 0, i = 1, 2, · · · , n. (2.3)

Let f (x) = −bx + ωg(x) + I, where x ∈ R. Assume that p1, p2, p1 < p2, are

the two points such that f0_(p

1) = f0(p2) = 0. Since 0 ≤ gj(ξ) ≤ 1 for all ξ ∈ R, we

have, for each i,

| n X j6=i ωijgj(xj) + Ji| ≤ n X j6=i |ωij| + |Ji| =: ki. Let ˆ fi(xi) = −bxi+ ωg(xi) + ki ˇ fi(xi) = −bxi+ ωg(xi) − ki.

Then ˇfi(xi) ≤ Fi(x) ≤ ˆfi(xi), for all x = (x1, · · · , xn) and for each i = 1, 2, · · · , n.

Condition (I): ˆfi(p1) < 0, ˇfi(p2) > 0, for i = 1, · · · , n.

Condition (II): −b + 1

4ε|ω| + ki < 0, for i = 1, · · · , n.

Theorem 2.1: Under Condition (I), there exist 3n _{equilibria for system (2.1) and}

(2.2). Under Conditions (I) and (II), there exist 2n _{asymptotically stable equilibria}

for system (2.1).

Proof: The existence of equilibrium follows from the Brouwer’s fixed point theorem. Indeed, under Condition (I), the function fi(xi) = −bixi+ωg(xi)+ki has three roots,

for every i. A mapping, defined on suitable region in the phase space, derived from solving (2.3) componentwise can be obtained. A fixed point of such a mapping gives rise to an equilibrium of (2.1) and (2.2), cf . [1].

For the second assertion, we observe that

DF (x) =      −b + ωg0_(x 1) ω12g0(x2) · · · ω1ng0(xn) ω21g0(x1) −b + ωg0(x2) ω2ng0(xn) ... ... . .. ... ωn1g0(x1) ωn2g0(x2) · · · −b + ωg0(xn)     . 3

Notably, max{g0_{(ξ) | ξ ∈ R} =} 1

4ε. Thus, under Condition (II), all the eigenvalues

of DF (x) for 2n _{equilibria x of system (2.1) have real parts less than zero, by the}

Gerschgorin’s Theorem.

References

[1] Chen, S.-S. and Shih, C.-W., Transversal homoclinic orbits in a transiently chaotic neural network, Chaos, 12, 654–670 (2002).

[2] Cheng, C.-Y., Chu, Y.-P. and Shih, C.-W., Stationary patterns and spatial entropy for spatially discrete reaction-diffusion equations, preprint, 2004.

[3] Forti, F., On global asymptotic stability of a class of nonlinear systems arising in neural network theory, J. Diff. Equations, 113, 246–264 (1994).

[4] Michel, A., Qualitative analysis of neural networks, IEEE Trans. Circuits Syst., 36 (1989), 229–243.

[5] Van Den Driessche, P. and Zou, X., Global attractivity in delayed hopfield neural network models, Siam. J. Appl. Math., 58(6) (1998), 1878–1890.

[6] Zhang, Yi, Estimate of exponential convergence rate and exponential stability for neural networks, IEEE Trans. Circuits Syst.

計畫成果自評： 我們推導了古典的類神經網路，與延遲性類神經網路多平衡解存在的條件； 並討論其在古典的神經網路系統之穩定性；並估計其 basin of attractions。目前， 我們正在做相關的數值模擬，希望瞭解 delays 在系統中對此類動態行為的影響。 相關的研究討論亦在熱烈進行中。這部份的成果預期在近期可發表於學術期刊。 在延遲性系統中，有關系統具多平衡態之動態收歛理論與實例，目前在文獻中， 仍相當缺乏；這一部份，也是我們努力的重點。在以類神經網路解組合最優化問 題的理論上，我們也已有一些想法來強化這個計算方法；這個想法仍是透過對系 統之動態行為更進一步的瞭解。 以上所述皆包含於原定計畫。對計畫中較難的課題，我們一方面學習，一方 面尋求可切入的研究點。我們已發掘出幾個具體的研究題目，希望很快有進一步 的發展。