c
World Scientific Publishing Company
ON THE DENSE ENTROPY OF TWO-DIMENSIONAL
INHOMOGENEOUS CELLULAR
NEURAL NETWORKS
JUNG-CHAO BAN∗
Department of Mathematics, National Hualien University of Education,
Hualien 97063, Taiwan [email protected]
CHIH-HUNG CHANG
Department of Applied Mathematics, National Chiao-Tung University,
Hsin-Chu 30050, Taiwan [email protected]
Received November 6, 2007; Revised December 10, 2007
This investigation elucidates the dense entropy of two-dimensional inhomogeneous cellular neural networks (ICNN) with/without input. It is strongly related to the learning problem (or inverse problem); the necessary and sufficient conditions for the admissibility of local patterns must be characterized. For ICNN with/without input, the entropy function is dense in [0, log 2] with respect to the parameter space and the radius of the interacting cells, indicating that, in some sense, ICNN exhibit a wide range of phenomena.
Keywords: Entropy; learning problem; ICNN.
1. Introduction
Cellular neural networks (CNN), as presented by Chua and Yang [1988a, 1988b] have been exten-sively investigated, and described in a review [Chua, 1998], which contains relevant references. Two of their applications are in image processing and pat-tern recognition. An important class of applications is steady-state solutions, including mosaic solutions and defect solutions [Chua, 1998; Hsu et al., 2000; Juang & Lin, 2000]. In recent years, the com-plexity of steady-state solutions has been exten-sively studied, and much attention has been paid to the complexity of the set of global patterns, with
particular reference to entropy [Ban et al., 2001a, 2001b, 2002; Ban & Lin, 2005; Ban et al., 2007a, 2007b; Chow et al., 1996a, 1996b; Hsu et al., 2000; Hsu & Yang, 2002; Juang & Lin, 2000; Lin & Shin, 1999; Lin & Yang, 2000, 2002; Lind & Marcus, 1995].
Two-dimensional (2-D) CNN is of the form,
dxi,j dt =−xi,j+ z + |k|,|l|≤d ak,lf (xi+k,j+l) + |k|,|l|≤d bk,lui+k,j+l, (1)
∗This work is partially supported by the National Science Council, R.O.C. under Grant No. 95-2115-M-026-003.
3221
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
where (i, j)∈ Z2, d∈ N, f(x) is a piecewise-linear output function, defined by
y = f (x) = 1 2(|x + 1| − |x − 1|). A = [ak,l] = a−d,d · · · ad,d .. . . .. ... a−d,−d · · · ad,−d and B = [bk,l] = b−d,d · · · bd,d .. . . .. ... b−d,−d · · · bd,−d
represent the feedback template and the controlling template, respectively; z denotes the biased term or threshold. The quantities xi,j denote the state at cell Ci,j, and yi,j denote the output at Ci,j.
As is generally known, stationary solutions x = (xi,j) are essential to understand CNN, and their outputs are called patterns. This study considers a specified class of output patterns called mosaic pat-terns. Ban et al. [2007b] investigated the connection between CNN with/without input and shift spaces, and an important question arose.
Problem. For CNN with/without input, if the radius of the interacting cells d is treated as a parameter, is {h(A, B, z, d)}/{h(A, z, d)} dense in [0, log 2]?
Multifractal analysis is introduced to a spec-ified dynamical system when one of its invariants is essentially the same as an interval (see [Pesin, 1997] for more detail), this motivates us to con-sider such a problem. However, since the well-known fact that the entropy of subshift of finite type takes a family of specific values, called Per-ron number [Lind & Marcus, 1995], the “dense” assumption cannot be removed. The main diffi-culty in solving the problem is related to the fact that the admissible local patterns that are pro-duced by CNN are very limited [Hsu et al., 2000; Juang & Lin, 2000]. Restated, there exists U ⊆
{1, −1}Zn×n such that U = B(A, z, d)/B(A, B, z, d) for all chosen values of the parameters A, B, z, d, where n = 2d + 1.
For example, consider the one-dimensional CNN without input, and the length of interac-tion d = 1. Figure 1 is the bifurcainterac-tion dia-gram that relates admissible local patterns to the parameters A = (al, a, ar) and z; readers may refer to [Hsu et al., 2000; Juang & Lin, 2000] for
Fig. 1. The bifurcation diagram of 1-D CNN.
more details. First, choosing (al, ar) yields a total of eight partitions, as shown in Fig. 1. Second, the (a−1, z) plane has 25 regions such that the admissi-ble local patterns will be uniquely determined once the region is chosen. For instance, if the parameters
A, z are in region [3, 4] of partition IV, the
admissi-ble local patterns are
B = {− ⊕ +, − ⊕ −, + ⊕ +, + −, + +, − −, − +}.
That is, “3” indicates that the three patterns with “+” in the center should be chosen from the bot-tom, and “4” indicates that all four patterns with “−” in the center can be chosen in IV. Thus, Figs. 1 and 2 show all admissible local patterns of 1-D CNN with d = 1.
However, letU ⊆ {1, −1}Z3×1 be the set of
pat-terns which are listed as follows.
U = {− ⊕ −, − ⊕ +, + ⊕ −, − −, − +, + −}.
Notably, U consists of patterns that are selected from different partitions for al and ar. More pre-cisely, the patterns with “+” in the center are located in partition V such that the parameters
al and ar must satisfy the conditions al < 0 and ar> 0. Moreover, the patterns with “−” in the
cen-ter are selected from partition I, in which the associ-ated parameters al, ar must then satisfy al, ar > 0.
Accordingly, there does not exist A, z such that
B(A, z) = U. Thus, some values of entropy cannot
be attained for all choices of 3× 1 basic sets for
d = 1.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
This study elucidates the ICNN system (defined later). Most materials in natural systems, including physical, biological and electronic systems, are
spatially inhomogeneous [Ferdinand & Fisher, 1969; Perdew, 1986; Rosenfeld, 1989; Kravtsov & Orlov, 1990; Pesin, 1997; Debye & Bueche, 2004]. The ICNN system is of the form,
dxi,j dt = −xi,j+ z + |k|,|l|≤d ak,lf (xi+k,j+l) + |k|,|l|≤d bk,lui+k,j+l, i, j≡ 0 mod m;
−xi,j+ z+ a0,0f (xi,j), otherwise,
(2)
for some m ∈ N, i, j ∈ Z. Restated, the difference between CNN and ICNN is that the templates and threshold at each cell Ci,j are spatially invariant for CNN but variant for ICNN. This work presents a solution to the problem of ICNN, but does not solve the problem of CNN. The authors suspect the answer to the problem of CNN is also positive.
In a work on dense entropy, Quas and Trow [2000] showed that every subshift of finite type (SFT) X with positive entropy has proper SFT X which is a subsystem of X whose entropy is strictly less than the entropy of X, but whose entropy is arbitrarily close to that of X. However, they cannot be guaranteed to be mixing [Quas & Sahin, 2003]. Recently, Desai [2006] proved that for anyZd-SFT
R of positive entropy, the SFT subsystems achieve
dense entropy in [0, h(R)]. Thus, if R is treated as a full shift, then the SFT is dense in [0, log|A|], where A denotes the symbols of R, and this result can be generalized to sofic systems. Restated, given
Fig. 2. The partition ofa − z plane of 1-D CNN.
aZdsofic shift T, the sofic shift subsystems achieve dense entropy in [0, h(T)]. However, a difficulty sim-ilar to that associated with CNN arises in solving the problem of ICNN. The difficulty is to guar-antee that the patterns that would achieve the desired entropy can be produced by an ICNN sys-tem with/without input. This investigation pro-poses a necessary and sufficient condition for the admissibility of local patterns of ICNN, and demon-strates that suitable local patterns can be found that achieve the given t ∈ [0, log 2] (according to Theorem A for ICNN without input and Theorem B for the case with input). Finding these patterns solves the dense entropy problem for ICNN.
The rest of this paper is organized as fol-lows. Section 2 introduces preliminaries that consti-tute the background for this work. Section 3 then presents a general theory that yields details about how ICNN relates to a shift of finite type. The solu-tion to the dense entropy problem is also addressed. Section 4 extends the results in Sec. 3 to ICNN with input.
2. Preliminary
Several notions for the formulation of the main results in Secs. 3 and 4 are presented in this sec-tion. Since the states Ci,j with i = k1m, j = k2m for k1, k2 ∈ Z are crucial for the study of the mosaic
solutions of ICNN, these cells are the main focus in the rest of this investigation.
Definition 2.1. Let x = (xi,j) be the stationary solution of system (2). x is called a mosaic solution if|xi,j| > 1 for all i, j ∈ Z, and is called an interior solution if |xi,j| < 1 for all i, j ∈ Z. A defect solu-tion x satisfies |xi,j| > 1 for some (i, j) ∈ D and |xk,| < 1 for some (k, ) /∈ D, where D Z2 and D= ∅.
First, considering the system (2) without input, that is, the template B≡ 0. For each given mosaic solution x, the output pattern at cell Ci,j is +, i.e.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
xi,j > 1, if and only if
|k|,|l|≤d (k, l)=(0,0)
ak,lyi+k,j+l+ a + z− 1 > 0, (3) where a0,0≡ a. Similarly, the output pattern at cell
Ci,j is −, i.e. xi,j <−1, if and only if
|k|,|l| ≤ d (k, l) = (0,0)
ak,lyi+k,j+l− a + z + 1 < 0. (4)
(3) and (4) can be rewritten in a much more com-pact form by introducing the following notations.
Denote n = 4d2 + 4d. Let Xn be the n-dimensional lattice points, i.e.
Xn={v = (vi)∈ Rn:|vi| = 1 for 1 ≤ i ≤ n}. (5) Then, for a given pair of template A and threshold
z, the basic set of admissible local patterns with
“+” state in the center is defined by
B(+, A, z, d) = {v ∈ Xn: α· v + a + z − 1 > 0}, where “·” is the inner product, α = (a1, a2, . . . , an), v = (v1, v2, . . . , vn) are obtained from
a4d2+2d · · · a2d2+d+1 · · · a1 .. . .. . · · · ai,j · · · ... .. . an · · · a2d2+3d · · · a2d+1 =
ai−d,j+d · · · ai,j+d · · · ai+d,j+d .. . .. . · · · ai,j · · · ... .. .
ai−d,j−d · · · ai,j−d · · · ai+d,j−d
and v4d2+2d · · · v2d2+d+1 · · · v1 .. . .. . · · · vi,j · · · ... .. . vn · · · v2d2+3d · · · v2d+1 =
yi−d,j+d · · · yi,j+d · · · yi+d,j+d
.. . .. . · · · yi,j · · · ... .. .
yi−d,j−d · · · yi,j−d · · · yi+d,j−d ,
respectively. In other words, α represents the sur-rounding template of A without center, and v indi-cates the output patterns at cell Ci,j whose center is omitted. Similarly, the basic set of admissible local patterns with “−” in the center is defined by
B(−, A, z, d) = {v ∈ Xn: α· v − a + z + 1 < 0}. An investigation of the basic sets of admissi-ble local patterns B(+, A, z, d) and B(−, A, z, d) is essential for the understanding of the global mosaic patterns on Z2 that are generated by the given (A, z). Some definitions and theorems should be stated first.
Definition 2.2. Given U ⊂ Xn,U is called separa-ble if there is a hyperplane H in Rn such that U andUc can be separated by H, whereUc = Xn\ U. Hsu et al. [2000] investigated how the admis-sible local mosaic patterns B(∗, A, z, d) relate to
the parameters A, z and d in CNN systems, where
∗ ∈ {+, −}.
Theorem 2.3 [Hsu et al., 2000]. There exists (A, z)
and d such that U = B(∗, A, z, d) for some ∗ ∈ {+, −} if and only if U is separable.
Moreover, the classical theory of convex set [Lay, 1992] gives the necessary and sufficient con-dition whenU ⊆ Xn is separable.
Theorem 2.4(Linear Separating Theorem). U and
Uc can be separated by a hyperplane in Rn if and only if
conv(U) ∩ conv(Uc) =∅, (6)
where conv(K) is the convex hull of K in Rn.
Let z = (z, z) denote the thresholds, and let
B(A, z)/B(A, B, z) denote the basic set of
admissi-ble local patterns of ICNN without/with input for
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
the given templates. The result in Theorem 2.3 still holds for ICNN systems.
Theorem 2.5. There exists (A, z) and d such that U = B(∗, A, z, d) for some ∗ ∈ {+, −} if and only if U ⊆ Xn is separable.
Proof. It suffices to show that B(+, A, z, d) = U for some (A, z) and d if and only if U is separable. The proof for∗ = − is essentially the same, thus is omitted.
First, considering the output pattern at Ci,j, where (i, j)= (k1m, k2m) for some k1, k2 ∈ Z. The
output pattern is + if and only if a + z − 1 > 0, and is− if and only if a − z− 1 > 0. Let a > 1 and
z = 1/2(a− 1). The output pattern at Ci,j can be arbitrary in such a case. It remains to show that U can be realized on Ci,j for some appropriate choice of (A, z), where i, j ≡ 0 mod m.
Let S = {U ⊆ Xn| U satisfies (6)}. For each U ∈ S, denoted by
A+(U) = {(α, p) | α · v + p > 0 for all v ∈ U}, (7)
A−(U) = {(α, q) | α · v + q < 0 for all v ∈ Uc}. (8) Then A+(U) ∩ A−(U) = ∅ if and only if U satisfies (6). In this case, the boundary ∂A+(U) of A+(U) consists of (A, B, z) such that α· v + a + z − 1 = 0, where p = a + z− 1.
Defining ˆ
B(+, α, p) = {v : α · v + p > 0}, (9) then ˆB(+, α, p) = U for all (α, p) ∈ A+(U). For each U ∈ S so that there exists (α, p) ∈ A+(U),
consider
z = p
2 − k, a = 1 +
p
2 + k, (10)
where k is chosen so that p/2 + k > 0. Then
B(+, A, z, d) = ˆB(+, α, p) = U, and vice versa. This
completes the proof.
Next, considering system (2) with input. Given a mosaic solution x, the output pattern at cell Ci,j is + if and only if |k|,|l|≤d (k, l)=(0,0) ak,lyi+k,j+l+ |k|,|l|≤d bk,lui+k,j+l + a + z− 1 > 0. (11)
Similarly, the output pattern at cell Ci,j is− if and only if |k|,|l| ≤ d (k, l) = (0,0) ak,lyi+k,j+l+ |k|,|l|≤d bk,lui+k,j+l − a + z + 1 < 0. (12)
It is seen from the above discussion that the basic set of admissible local patterns with “+” in the center is defined by
B(+, A, B, z, d) = {(v, w) ∈ Xn× Xn+1: α· v + β· w + a + z − 1 > 0}, and the basic set of admissible local patterns with “−” in the center is defined by
B(−, A, B, z, d) = {(v, w) ∈ Xn× Xn+1: α· v + β· w − a + z + 1 < 0}. Herein, β = (b1, b2, . . . , bn+1) and w = (w1, w2, . . . , wn+1) are obtained from
b4d2+2d+1 · · · b2d2+d+1 · · · b1 .. . .. . · · · b2d2+2d+1 · · · ... .. . bn+1 · · · b2d2+3d+1 · · · b2d+1 =
bi−d,j+d · · · bi,j+d · · · bi+d,j+d
.. . .. . · · · bi,j · · · ... .. .
bi−d,j−d · · · bi,j−d · · · bi+d,j−d
and w4d2+2d+1 · · · w2d2+d+1 · · · w1 .. . .. . · · · w2d2+2d+1 · · · ... .. . wn+1 · · · w2d2+3d+1 · · · w2d+1 =
ui−d,j+d · · · ui,j+d · · · ui+d,j+d
.. . .. . · · · ui,j · · · ... .. .
ui−d,j−d · · · ui,j−d · · · ui+d,j−d
,
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
respectively. Namely, β represents the template B and w indicates the input patterns at cell Ci,j.
Ban et al. [2007b] generalized Theorem 2.3 to a common case that the controlling template B is considered.
Theorem 2.6[Ban et al., 2007b]. There exists (A, B, z) and d such that U = B(∗, A, B, z, d) for
some∗ ∈ {+, −} if and only if U is separable.
Theorem 2.6 can also be applied for ICNN with input.
Theorem 2.7. There exists (A, B, z) and d such that U = B(∗, A, B, z, d) for some ∗ ∈ {+, −} if and only if U is separable.
Proof. This can be accomplished via analogous method as in the proof of Theorem 2.5, thus is
omitted.
3. Inhomogeneous Cellular Neural Networks without Input
The dense entropy property for the ICNN without input is studied in this section. Section 3.1 develops the fundamental theory and presents its application for ICNN in Sec. 3.2.
3.1. Two-dimensional subshift of finite type
This subsection investigates the preliminaries that are necessary for the understanding of dense entropy property of ICNN without input.
Definition 3.1. Let X ⊆ {1, −1}Z2 be a two-dimensional shift space with finite alphabet
A(X) = {1, −1}.
(1) If x∈ X and S ⊆ Z2, the restriction of x to S is denoted by πS(x).
(2) Let Λ(n) ={(p, q) : p, q ∈ Z, 0 ≤ p, q ≤ n − 1}. An n-block is πc+Λ(n)(x) for some c ∈ Z2,
x∈ X. The set of n-blocks is denoted by Bn(X). (3) A configuration on S ⊆ Z2 is a map E : S →
A(X). For x ∈ X, E occurs in x if πc+S(x) = E
for some c ∈ Z2.
(4) For each c ∈ Z2, the shift map σc : X → X is defined by πd(σc(x)) = πc+d(x) for all d ∈ Z2. Moreover, the iteration of σc is denoted by
σc= σc◦ σc−1 for all ∈ N.
Denote πΛ(n)(x) by πn(x) for simplicity.
Definition 3.2. Given U ⊆ {1, −1}Zn×n, s∈ N, s <
n, the shift space Xs(U) ⊆ {1, −1}Z
2 is defined by Xs(U) = {x ∈ {1, −1}Z 2 : πn(σ(i,j)(x))∈ U for all ∈ Z, i, j ∈ {0, n − s}}. (13) Moreover, the r-copy ofU, Ur ⊆ {1, −1}Zk×k, where
k = rn− (r − 1)s, is defined by
Ur={v ∈ {1, −1}Zk×k :∃ x ∈ X
s(U)
such that πk(x) = v}. (14)
Remark 3.3. In other words, Ur consists of those patterns combined by r2-many patterns in U with
s-many rows/columns overlapped. For example,
consider U ⊆ {1, −1}Z4×4 and s = 1. U2 consists
of those patterns with size 7× 7 such that each pat-tern v ∈ U2 is a combination of four patterns in U with one-row/column overlapped. As seen in Fig. 3, the last column on the right-hand side in pattern 1 can be overlapped with the first column on the left-hand side in pattern 2 if and only if these two 1× 4 patterns are exactly the same. The same applies to the top row in pattern 1 and the bottom row in pattern 3.
Next, the effect of the parameter s is studied. In general, the range of s is less than n and greater than one. After constructing Ur from a given U, the lemma below studies the relationship between the subshifts of finite types Xs(U) and Xs(Ur). In addition, it reduces the complexity caused by s.
Lemma 3.4. Given U ⊆ {1, −1}Zn×n and r ∈ N,
then Xs(U) = Xs(Ur).
Proof. Since Ur is constructed from U such that each pattern in Ur consists of r2-many patterns in
U with s-many columns/rows overlapped, it is seen
that Xs(Ur) ⊆ Xs(U). This remains to show that
Xs(U) ⊆ Xs(Ur).
If x ∈ Xs(U), then πn(σ(i,j) (x)) ∈ U for all
∈ Z, where i, j ∈ {0, n − s}. Definition 3.2 shows
that πk(x) ∈ Ur, where k = rn− (r − 1)s − 1. Let y = σ(i,j)(x) for some i, j ∈ {0, n − s}. Then
πk(y)∈ Ur via the same argument. It can be easily checked that πk(σ(i,j) (x))∈ U for all ∈ Z by math-ematical induction, where i, j ∈ {0, n − s}. There-fore, x∈ Xs(Ur) and this completes the proof. Without loss of generality, assuming that s ≤ [n/2], where [·] is the Gauss function. The case where s > [n/2] is discussed in Remark 3.8.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
1
2
1
2
3
3
4
4
(a) (b)Fig. 3. The construction ofUrfor a givenU and r ∈ N. Take U ⊆ {1, −1}Z4×4,r = 2 and s = 1 as an example. First pick
four patterns inU, say P1, P2, P3, P4. If the patterns in the first row ofP1differ from the patterns in the last row ofP3, then nothing happens. Otherwise, P1 andP3 are combined with one-row overlapped. Repeating this process, a new pattern with size 7× 7 is thus derived.
It is seen so far that a subshift of finite type is generated onceU and s are given. The method that embeds a chosen set of admissible local patterns in an ICNN system is introduced.
If U ⊆ {1, −1}Zn×n is given and n is
even, then an extension of U, denoted by
V ⊂ {1, −1}Z(n+1)×(n+1), is constructed as follows.
v = (v(i,j))∈ V if and only if
(i) v(i,j)=−1 if i = (n/2) + 1 or j = (n/2) + 1. (ii) v(n/2)+1 = u for some u ∈ U, where vp;q ∈
{1, −1}Zn×n is obtained from v by deleting row
p and column q, and denoted by vp if p = q.
Similarly, if n is odd, constructing V ⊂
{1, −1}Z(n+2)×(n+2) by v = (v(i,j)) ∈ V if and
only if
(i) v(i,j) = −1 if either i or j ∈ {(n + 1)/2, (n + 3)/2}.
(ii) v(n−1)/2 = u for some u ∈ U, where vp;q ∈
{1, −1}Zn×n is obtained from v
p;q by deleting row p and column q, and denoted by vp if
p = q.
More precisely, U is extended to V by adding a cross of “−1” to the center of each u ∈ U. Under such extension, there is a one-to-one correspon-dence between U and V. Figure 4 gives two exam-ples for the cases where n is odd and n is even, respectively.
Remark 3.5. Notably, the size ofV is odd no matter what the size of U is. That is, V ⊆ {1, −1}Z× for some = 2k + 1, k∈ N.
For each U ⊆ {1, −1}Zn×n, there associates an unique V ⊆ {1, −1}Z(n+1)×(n+1) under the
con-struction above. The relationship between Xs(U) and Xs(V) is investigated below. Before stating the lemma, a definition is given first.
Definition 3.6. Let X, Y be shift spaces with shift maps σX and σY, respectively. Define φ : X → Y
be a factor map from X to Y if φ is onto and
φ◦ σX = σY ◦ φ. X is conjugate to Y, denoted
by X ∼=Y, if φ is a factor map and one-to-one.
A key lemma then follows.
Lemma 3.7. Given U ⊆ {1, −1}Zn×n, constructing
V as above, then Xs(U) ∼=Xs(V).
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
(a)
(b)
Fig. 4. (a) Extend a 4× 4 pattern to a 5 × 5 pattern by adding a cross of pattern into the center of the original one. The pattern “+” is represented by red and the pattern “−” is represented by white and blue, herein blue is used to distinguish from the original pattern. (b) Extend a 5× 5 pattern to a 7 × 7 pattern.
Proof. Define ψ :V → U by ψ(v) = u, where
u = vn 2+1, n is even; v n−1 2 , n is odd. (15) For simplicity, assume n is even. The case where n is odd can be done similarly. It is easily seen that ψ(v) is bijective. Furthermore, defining φ : Xs(V) →
Xs(U) by φ(y)(i,j)+c = ψ(πn+1(y(qi,qj)))c, where
i, j∈ {0, n − s}, ∈ Z, q = (n − s + 1)/(n − s), c ∈
Λ(n) and y∈ Xs(V). In such a case, φ ◦ σXs(V)=
σXs(U)◦φ and φ is a conjugacy since ψ is one-to-one
and onto. This completes the proof.
Remark 3.8. If s > [n/2], let ∈ N satisfy (− 1)(n − s) + s 2 < s≤ (n− s) + s 2 . (16) Then constructV via the same method mentioned above so that there is a one-to-one correspondence between V and U. Similar as above, Lemmas 3.4 and 3.7 show that Xs(U) ∼=Xs(V).
3.2. Two-dimensional inhomogeneous cellular neural networks
without input
Section 3.1 shows that Xs(U) = Xs(Ur) and
Xs(U) ∼= Xs(V), where U ⊆ {1, −1}Zn×n is given, V ⊆ {1, −1}Z(n+1)×(n+1) is obtained from U and
r ∈ N. This subsection applies the theory
devel-oped in the last subsection to ICNN without input. First, the preservation of the separation property betweenU and V is given below.
Lemma 3.9. Given U ⊆ {1, −1}Zn×n, then U is
separable if and only if V is separable.
Proof. For simplicity, the case where n is even is proved. It can be done similarly when n is odd.
If U is separable, there is a linear functional
g : {1, −1}Zn×n → R and α ∈ R so that g(u) < α for all u ∈ U, and g(u) > α for all u ∈ Uc. Let ρ = α − min{g(u) : u ∈ U}. Define ˆg :
{1, −1}Z(n+1)×(n+1) → R by
ˆ
g(v) = g(u) + ρ i or j=(n/2)+1
v(i,j), (17)
where u∈ {1, −1}Zn×n is obtained from v by delet-ing row ((n/2) + 1) and column ((n/2) + 1). Then ˆ
g(v) < α − (2n + 1)ρ for v ∈ V and ˆg(v) > α− (2n + 1)ρ for v ∈ Vc. Thus, V is separable.
Similarly, if V is separable, then so is U. This completes the proof.
Before stating the main theorem, the following theorem is essential for the study of the mosaic solu-tions of ICNN.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
Theorem 3.10. GivenU ⊆ {1, −1}Zn×n, and s∈ N. If U is separable, then there exist m ∈ N and (A, z, d)
for system (2) such that X(B(A, z, d)) ∼=Xs(U), where B(A, z, d) is the admissible local patterns obtained from (2) with parameters (A, z, d),
X(B(A, z, d)) = x∈ {1, −1}Z2 : πκ(d)(σ (i,j)(x))∈ B(A, z, d) for all ∈ Z, i, j ∈ {0, m} , (18) and κ(d) ={(p, q) : −d ≤ p, q ≤ d, p, q ∈ Z}.
Proof. Without loss of generality, assume that n is even and s≤ n/2. Once U is given, construct V as above. Lemmas 3.7 and 3.9 indicate that Xs(V) ∼=
Xs(U) and V is separable. Consider d = n/2, The-orem 2.5 shows that there exists (A, z, d) so that
B(A, z, d) = V.
Let m = 2d− s + 1. For each x ∈ X(B(A, z, d)), (18) implies
πκ(d)(x(i,j))∈ B(A, z, d) for all ∈ Z, i, j ∈ {0, n − s + 1}.
It is easily seen that X(B(A, z, d)) = Xs(V). Since
Xs(U) ∼=Xs(V), the proof is completed. When (A, z, d) is given, the basic set of admis-sible local patterns B = B(A, z, d) is immedi-ately determined. Let Σp,q(X(B)) denote the set of global patterns in X(B) with size p × q, and let Γp,q(X(B)) = |Σp,q(X(B))|. The entropy of X(B) is defined by h(X(B)) ≡ lim p,q→∞ log Γp,q(X(B)) pq .
The existence of the limit can be found in [Chow
et al., 1996b].
The first main theorem of this investigation, the dense entropy property of ICNN without input, is as follows.
Theorem A. For t ∈ [0, log 2], ε > 0, there exist m ∈ N and (A, z, d) such that |h(X(B(A, z, d))) − t| < ε.
Before proving the theorem, the following lem-mas should be stated first.
Lemma 3.11. Let Sn,l⊂ Xn be defined by Sn,l={x = (x1, . . . , xn)∈ Xn: xk=−1
for all l + 1≤ k ≤ n}, (19) 1 ≤ l ≤ n − 1, and Sn,n = Xn. Then Sn,l is separable.
Proof. Define a linear functional g :Rn→ R by
g(x) = n i=l+1
xi for all x = (xi)ni=1∈ Rn. (20) Let h(x) = g(x)+(n−l−1). It can be easily checked that h(x) < 0 for all x∈ Sn,l and h(x) > 0 for all x∈ Sn,lc . That is, Sn,l and Sn,lc can be separated by the hyperplane
H ={x ∈ Rn: g(x) = l− n + 1}. (21) This completes the proof.
Theorem 3.12. Given l, d∈ N and n = 4d2. There exists Ud,l ⊆ {1, −1}Z2d×2d such that h(X
d(Ud,l)) = (l/n) log 2 and Ud,l is separable, where 1≤ l ≤ n.
Proof. If d, l ∈ N is given, n = 4d2 and 1≤ l ≤ n. Define
T :{1, −1}Zn×1 → {1, −1}Z2d×2d
by
(T ν)i,j = ν2d(i−1)+j
for all ν = (νk)∈ {1, −1}Zn×1. (22) Let Sn,l be defined as in Lemma 3.11, and let Mn,l be defined as follows.
Mn,l ≡ {K ∈ {1, −1}Z2d×2d :∃ ν ∈ S
n,l such that K = T ν}.
Furthermore, constructUd,l ⊆ {1, −1}Z4d×4d as
fol-lows. J ∈ Ud,l if π(i,j)+Λ(2d)(J ) ∈ Mn,l for i, j ∈
{0, 2d}, where Λ(n) is defined as in Definition 3.1.
Claim.Ud,l is separable.
Let g : Rn → R be defined as in (20) and ˜
g = g◦ T−1. For w ∈ {1, −1}Z4d×4d, rewriting w
as w = w1 w2
w3 w4 , where wi ∈ {1, −1}
Z2d×2d for all i.
Define a linear functional τ : {1, −1}Z4d×4d → R
by τ (w) = ˜g(w1) + ˜g(w2) + ˜g(w3) + ˜g(w4) and ˜
τ (w) = τ (w) + 4n− 4l − 1. The above
constitu-tion confirms that ˜τ (w) < 0 for all w ∈ Ud,l and ˜
τ (w) > 0 otherwise. That meansUd,l is separable.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
Moreover, there are 2l-many patterns for each block of x∈ Xd(Ud,l) with size 2d× 2d. Therefore,
h(Xd(Ud,l)) = lim p,q→∞ log Γ2dp·2dq(Xd(Ud,l)) 2dp· 2dq = lim p,q→∞ log(2l)pq 4d2pq = l nlog 2.
This completes the proof.
Proof of Theorem A. For t ∈ [0, log 2] and ε > 0, there exist d, l ∈ N such that |(l/n) log 2 − t| <
ε, where n = 4d2. Theorem 3.12 indicates there is a separable set Ud,l such that h(Xd(Ud,l)) = (l/n) log 2. Lemma 3.7, Lemma 3.9 and Theorem 3.10 show h(Xd(Ud,l)) = h(Xd(Vd,l)) and there exist
m∈ N and (A, z, d) such that B(A, z, d) = Vd,l. The proof is then completed.
4. Inhomogeneous Cellular Neural Networks with Input
In this section, Theorem A is extended to the case where B is not identical to zero.
Once the parameters (A, B, z, d) are given, the basic set of admissible local patterns is determined and denoted by
B ≡ B(A, B, z, d)
={Y ◦ U} ⊆ {1, −1}Z(2d+1)×(2d+1)×2,
where Y, U ∈ {1, −1}Z(2d+1)×(2d+1). The output
pat-tern Y coupled with input patpat-tern U , denoted by
Y◦U, is a two-layer array. Defining the output space
generated byB(A, B, z, d) as follows.
X(B) =
y∈ {1, −1}Z2 : there exists u∈ {1, −1}Z2 such that
πκ(d)(σ(i,j)(y◦ u)) ∈ B for all ∈ Z, i, j ∈ {0, m}
, (23)
where κ(d) is defined in Theorem 3.10 and
πκ(d)(σ(i,j)(y◦u)) ≡ πκ(d)(σ(i,j)(y))◦πκ(d)(σ(i,j)(u)). For d, l ∈ N, let Ud,l be the same as defined in the proof of Theorem 3.12. Denote by
Vd,l={Y ◦ U : Y, U ∈ Ud,l} ⊆ {1, −1}Z2d×2d×2. (24) Then the lemma follows.
Lemma 4.1. Vd,l is separable.
Proof. Let τ be the same as in the proof of Theorem 3.12. Define a linear functional θ : {1, −1}Z4d×4d ×
{1, −1}Z4d×4d → R by θ(u, v) = τ(u) + τ(v) and
θ(u, v) = θ(u, v) + 8n + 8l − 1. It is easily checked that θ(u◦ v) < 0 for all u ◦ v ∈ Vd,land θ(u◦ v) > 0 otherwise. This completes the proof.
Furthermore, the entropy of the subshift space induced by Vd,l can be computed via the same method as in the proof of Theorem 3.12, thus the proof is omitted.
Theorem 4.2. h(Xd(Vd,l)) = (l/n) log 2.
The dense entropy property of ICNN with input then follows.
Theorem B. For t ∈ [0, log 2], ε > 0, there exist m ∈ N and (A, B, z, d) such that |h(X(B(A, B,
z, d))) − t| < ε.
The proof of Theorem B can be accomplished via the same discussion in the proof of Theorem A, hence is skipped.
Acknowledgment
The authors thank Prof. Song-Sun Lin for some helpful discussions and suggestions.
References
Ban, J.-C., Chien, K.-P. & Lin, S.-S. [2001a] “Spatial disorder of CNN-with asymmetric output function,”
Int. J. Bifurcation and Chaos 11, 2085–2095.
Ban, J.-C., Lin, S.-S. & Shih, C.-W. [2001b] “Exact number of mosaic patterns in one-dimensional cellular neural networks,” Int. J. Bifurcation and Chaos 11, 1645–1653.
Ban, J.-C., Hsu, C.-H. & Lin, S.-S. [2002] “Spatial disor-der of cellular neural network-with biased term,” Int.
J. Bifurcation and Chaos 12, 525–534.
Ban, J.-C. & Lin, S.-S. [2005] “Patterns generation and transition matrices in multi-dimensional lattice models,” Discr. Contin. Dyn. Syst. 13, 637–658.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
Ban, J.-C., Lin, S.-S. & Lin, Y.-H. [2007a] “Patterns gen-eration and spatial entropy in two-dimensional lattice models,” Asian J. Math. 11, 497–534.
Ban, J.-C., Chang, C.-H., Lin, S.-S. & Lin, Y.-H. [2007b] “Entropy and zeta function for one-dimensional multi-layer cellular neural networks,” preprint. Chow, S.-N., Mallet-Paret, J. & Van Vleck, E. S. [1996a]
“Dynamics of lattice differential equations,” Int. J.
Bifurcation and Chaos 6, 1605–1621.
Chow, S.-N., Mallet-Paret, J. & Van Vleck, E. S. [1996b] “Pattern formation and spatial chaos in spatially dis-crete evolution equations,” Rand. Comput. Dynam. 4, 109–178.
Chua, L.-O. & Yang, L. [1988a] “Cellular neural net-works: Theory,” IEEE Trans. Circuits Syst. 35, 1257–1272.
Chua, L.-O. & Yang, L. [1988b] “Cellular neural net-works: Applications,” IEEE Trans. Circuits Syst. 35, 1273–1290.
Chua, L.-O. [1998] CNN: A Paradigm for
Complex-ity, World Scientific Series on Nonlinear Science,
Series A, Vol. 31 (World Scientific, Singapore). Debye, P. & Bueche, A. M. [2004] “Scattering by an
inho-mogeneous solid,” J. Appl. Phys. 20, 518–525. Desai, A. [2006] “Subsystem entropies for Zd sofic
sys-tems,” Indagationes Mathematicae 17, 353–359. F¨ath, M., Freisem, S., Menovsky, A. A., Tomioka, Y.,
Aarts, J. & Mydosh, J. A. [1999] “Spatially inhomo-geneous metal-insulator transition in doped mangan-ites,” Science 285, 1540–1542.
Ferdinand, A. E. & Fisher, M. E. [1969] “Bounded and inhomogeneous ising models. I. Specific-heat anomaly of a finite lattice,” Phys. Rev. 185, 832–846.
Hsu, C.-H., Juang, J., Lin, S.-S. & Lin, W.-W. [2000] “Cellular neural networks: Local patterns for general template,” Int. J. Bifurcation and Chaos 10, 1645– 1659.
Hsu, C.-H. & Yang, T.-H. [2002] “Abundance of mosaic patterns for CNN with spatially variant templates,”
Int. J. Bifurcation and Chaos 12, 1321–1332.
Juang, J. & Lin, S.-S. [2000] “Cellular neural networks: Mosaic pattern and spatial chaos,” SIAM J. Appl.
Math. 60, 891–915.
Kravtsov, Y. A. & Orlov, Y. I. [1990] Geometrical
Optics of Inhomogeneous Media (Springer-Verlag,
Berlin Heidelberg, NY).
Lay, R. [1992] Convex Sets and Their Applications (Wiley, NY).
Lin, S.-S. & Shih, C.-W. [1999] “Complete stability for standard cellular neural networks,” Int. J. Bifurcation
and Chaos 9, 909–918.
Lin, S.-S. & Yang, T.-S. [2000] “Spatial entropy of one-dimensional celluar neural network,” Int. J.
Bifurca-tion and Chaos 10, 2129–2140.
Lin, S.-S. & Yang, T.-S. [2002] “On the spatial entropy and patterns of two-dimensional cellular neural net-works,” Int. J. Bifurcation and Chaos 12, 115–128. Lind, D. & Marcus, B. [1995] An Introduction to
Sym-bolic Dynamics and Coding (Cambridge University
Press, Cambridge).
Perdew, J. P. [1986] “Density-functional approximation for the correlation energy of the inhomogeneous elec-tron gas,” Phys. Rev. B 33, 8822–8824.
Pesin, Y. [1997] Dimension Theory in Dynamical
Sys-tems (University of Chicago Press).
Quas, A. & Trow, P. [2000] “Subshifts of multi-dimensional shifts of finite type,” Erg. Th. Dyn. Syst.
20, 859–874.
Quas, A. & Sahin, A. [2003] “Entropy gaps and locally maximal entropy inZdsubshifts,” Erg. Th. Dyn. Syst.
23, 1227–1245.
Rosenfeld, Y. [1989] “Free-energy model for the inho-mogeneous hard-sphere fluid mixture and density-functional theory of freezing,” Phys. Rev. Lett. 63, 980–983.
Int. J. Bifurcation Chaos 2008.18:3221-3231. Downloaded from www.worldscientific.com
