with Lipschitz constant Lj, then DRNN (1.1) has a unique solution for every given initial condition.
Hence, each Fi is Lipschitz, and then F in Theorem 2.1.3 is Lipschitz. Consequently, (1.1) has a unique solution for any given initial condition. ¤
2.3 Lyapunov Functional and Lyapunov-Razumikhin Theorem
In the case of ordinary differential equations (2.3), complete stability (convergence) and quasiconvergence could be based on applying the LaSalle’s invariant principle to the Lyapunov functions. Let us review this principle quoted from [23].
Suppose the vector field F in (2.3) is locally Lipschitzian. Let L be a scalar function defined and continuous on Rn and ϕ(t, x) be the flow map of (2.3). To determine if L decreases along the orbit of (2.3), we can consider
˙L(x) := lim sup
h→0+
1
h[L(ϕ(h, x)) − L(x)]. (2.4)
If L is locally Lipschitz continuous, (2.4) is equal to lim sup
h→0+
1
h[L(x + hF(x)) − L(x)]. (2.5)
Suppose L is bounded in Rnand ˙L(x) ≤ 0 for all x ∈ Rn. Let E := {x ∈ Rn| ˙L(x) = 0}
and let M be the largest invariant set of (2.3) in E. LaSalle’s invariant principle says that if ϕ(t, x) is bounded for t ≥ 0, then the ω-limit set of ϕ(t, x) belongs to M.
There also exists an analogous theory in delayed equations. Consider the DDE (2.2)
dx(t)
dt = F (xt), where F : C → Rn is completely continuous.
Definition 2.3.1. We say W : C → R is a Lyapunov functional on a set S in C relative to (2.2) if W is continuous on ¯S, the closure of S, and ˙W ≤ 0 on S, where
W (φ) := lim sup˙
h→0+
1
h[W (xh(φ)) − W (φ)]. (2.6) For the given S, let
E(S) := {φ ∈ ¯S| ˙W (φ) = 0}
and let M(S) denote the largest subset of E(S) that is invariant under the flow gen-erated by Eq (2.2). The following theorem is an invariant principle for autonomous delayed differential equations.
Theorem 2.3.2. [25] If W is a Lyapunov functional on S and xt(φ) is a bounded solution of Eq (2.2) that remains in S, then xt(φ) tends to M(S) as t → ∞.
The following result is concerned with the stability of a system with a single equilibrium.
Corollary 2.3.3. [25] Suppose W : C → R is continuous and there exist nonnegative continuous functions a(·) and b(·), a(0) = b(0) = 0, limr→+∞a(r) = +∞ and
a(|φ(0)|) ≤ W (φ), W (φ) ≤ −b(|φ(0)|).˙
Then the trivial solution is stable and every solution is bounded. If, in addition, b(·) is positive definite, then every solution approaches the trivial solution as t → ∞.
Another approach for studying the stability of steady states in a delayed dif-ferential equations is constructing an appropriate Lyapunov “function” for the given system.
We say V : Rn → R is a Lyapunov function (or Razumikhin function) if V has continuous first partial derivatives. For a Lyapunov function, we define the upper right-hand derivative of V with respect to (2.2) is defined as
V (φ) := lim sup˙
The second equality holds when V has continuous first partial derivatives. For a given set S ⊆ C, define
E(S) := {φ ∈ ¯˜ S| max
−τ ≤θ≤0V (xt(φ)(θ)) = max
−τ ≤θ≤0V (φ(θ)) for all t ≥ 0}
and let ˜M(S) denote the largest subset of ˜E(S) that is invariant under the flow gen-erated by Eq (2.2). The following theorem is an invariance principle for autonomous delayed differential equations.
Theorem 2.3.4. [21] Suppose there exist a Lyapunov function V and a closed set S in C that is positively invariant under Eq (2.2) such that
V (φ) ≤ 0, for all φ ∈ S with V (φ(0)) = max˙
−τ ≤θ≤0V (φ(θ)).
Then for any φ ∈ S such that x(φ)(·) is defined and bounded on [−τ, ∞), ω(φ) ⊆ M(S) ⊆ ˜˜ E(S). Hence xt(φ) → ˜M(S) as t → ∞.
As an consequence of Theorem 2.3.4, the following is an asymptotic stability of an equilibrium for autonomous delayed differential equations.
Corollary 2.3.5. [21] Let F (0) = 0 and suppose there exist a Lyapunov function V and a constant α > 0 such that
(i) V (0) = 0 and V (φ) > 0 f or all 0 6= kφk < α, (ii) V (0) = 0, and˙
(iii) V (φ) < 0 f or all 0 6= kφk < α with max˙
−τ ≤θ≤0V (φ(θ)) = V (φ(0)).
Then the solution x = 0 of Eq (2.2) is asymptotically stable.
The LaSalle’s invariant principle is an effective methodology to investigate the stability of steady states and global dynamics. However, suitable Lyapunov functions or Lyapunov functionals need to be constructed to fit the practical models. Moreover, let us recall that the functional W is defined on the infinite dimensional Banach space C and the definition (2.7) is concerned with the functional F . From the definitions (2.5), (2.6) and (2.7), we know that it is more difficult to propose a Lyapunov functional W or a Lyapunov function V with negative derivative along solutions of the delayed differ-ential equation (2.2) than to construct a Lyapunov function in the ordinary differdiffer-ential equation (2.3).
Chapter 3
Monotone Dynamical Systems
In [27, 28], Hirsch developed a theory on almost quasiconvergence in continuous time networks. In such a dynamical scenario, there may exist cycles or other kinds of non-convergent orbits, but they cannot be stable. We will employ the monotone dynamics theory to explore the almost quasiconvergence of delayed recurrent neural networks in Chapter 5. Monotone dynamics theory has been widely applied in systems including reaction-diffusion systems, semilinear diffusion equations and various biological sys-tems. Matano introduced the important idea of strongly order preserving semiflows [38], which is more flexible than strong monotonicity, proposed by Hirsch. The work of Smith and Thieme [47, 48] represents a synthesis of the approaches of Hirsch and Matano that attempts to simplify and streamline the arguments. Significant improve-ments in the theory was obtained therein with additional compactness hypotheses that are often satisfied in the applications.
In this chapter, we recall some notations and basic theory of monotone dynamical systems from [47]. In Chapter 5, we will further confirm that quasiconvergence is generic for the networks through justifying the strongly order preserving property as the self-feedback time lags are small by using the theory of Smith and Thieme [48].
3.1 Preliminary
In this section, we introduce the basic theory of monotone dynamical systems which will be applied to study the convergence of dynamics in the topic of neural networks.
Consider an ordered metric space Ω with metric d and partial order relaton ≤ which means that:
(i) x ≤ x for all x ∈ Ω (reflexive);
(ii) x ≤ y and y ≤ z implies x ≤ z (transitive);
(iii) x ≤ y and y ≤ x implies x = y (antisymmetric).
Definition 3.1.1. (i) We write x < y if x ≤ y and x 6= y.
(ii) Given subsets U and V of Ω, we write U ≤ V (U < V ) when x ≤ y(x < y) holds for each choice of x ∈ U and y ∈ V .
We assume that the partial order relation is closed; it means that the order relation and the topology on Ω are compatible in the sense that x ≤ y whenever xn → x and yn → y as n → ∞ and xn ≤ yn for all n. For A ⊂ Ω we write A for the closure of A and IntA for the interior of A.
In the applications, the order relation usually comes from a positive cone. It means that Ω is typically a subset of a Banach space ˜Ω with a nonempty closed subset, positive cone, K possessing the properties :
(i) R+· K ⊂ K, (ii) K + K ⊂ K, (iii) K ∩ (−K) = {0},
where R+ := (0, +∞) and −K := {−k|k ∈ K}. In this case, the relation defined by x ≤ y if and only if y − x ∈ K is a closed partial order relation.
Definition 3.1.2. (I) A semif low on Ω is a continuous map Φ : Ω × R+ → Ω which satisfies :
(i) Φ0 = idΩ
(ii) Φt◦ Φs = Φt+s f or t, s ≥ 0.
Here, Φt(x) := Φ(x, t) for x ∈ Ω and idΩ is the identity map on Ω.
(II) The orbit of x is denoted by
O(x) := {Φt(x)|t ≥ 0}.
Definition 3.1.3. Let E be the set of all equilibrium points for Φ. (i) The omega limit set, ω(x), of x ∈ Ω is defined by
ω(x) = ∩t≥0∪s≥tΦs(x).
(ii) A point x ∈ Ω is called a quasiconvergent point if ω(x) ⊂ E. The set of such points is denoted by Q.
(iii) A point x ∈ Ω is called a convergent point if ω(x) consists of a single point of E.
The set of such points is denoted by C.
Definition 3.1.4. (i) The semiflow Φ is said to be monotone provided Φt(x) ≤ Φt(y) whenever x ≤ y and t ≥ 0.
(ii) Φ is called strongly order preserving, SOP, if it is monotone and whenever x < y there exist open subsets U, V of Ω with x ∈ U and y ∈ V and t0 > 0 such that
Φt0(U) ≤ Φt0(V ).
Note that monotonicity of Φ implies that Φt(U) ≤ Φt(V ) for all t ≥ t0. A dynamical system on Ω is monotone if it preserves the ordering of initial data. A SOP system has stronger ordering preserving about the neighborhoods of two points, x < y.
The order relation between these two points, x < y, will be kept forever.