4 Structures in Multi-layer Cellular Neural Net- Net-works

The propositions and theorems we obtain in previous sections still hold for general multi-layer neural networks. In this section, we only state the results instead of giving their explicit proofs.

4.1 Two-layer cellular neural networks

First we consider a two-layer cellular neural network whose basic set of admissible local patterns are of size k × 2, k≥ 3. For simplicity, we assume that each parameter is nonzero.

Y⁽¹⁾ Y⁽²⁾ Y⁽³⁾ ⋯⋯⋯ Y⁽ⁿ⁻¹⁾ Y⁽ⁿ⁾ W₁⁽¹⁾ W₂⁽¹⁾ W₃⁽¹⁾ ⋯⋯⋯ W_n−1⁽¹⁾ W_n⁽¹⁾

W₁⁽²⁾ W₂⁽²⁾ ⋯⋯⋯⋯⋯ W_n−1⁽²⁾

W₁⁽³⁾ W_n−2⁽³⁾

W

⋯⋯⋯⋯⋯⋯⋯⋯

Figure 11: A common extension space of Y⁽¹⁾, Y⁽²⁾, . . . , Y⁽ⁿ⁾can be obtained iteratively.

Suppose that k = 3, we embed our problem in the case that k = 4. In other words, suppose B⊆ {−,+}^Z3×2. Let

B=⎧⎪⎪⎪

⎨⎪⎪⎪⎩

y⁽²⁾₀ y⁽²⁾₁ y⁽²⁾₂ y⁽²⁾₃

y⁽¹⁾₀ y⁽¹⁾₁ y⁽¹⁾₂ y⁽¹⁾₃ ∶ y₀⁽²⁾y₁⁽²⁾y₂⁽²⁾

y₀⁽¹⁾y₁⁽¹⁾y₂⁽¹⁾ , y⁽²⁾₁ y⁽²⁾₂ y⁽²⁾₃ y⁽¹⁾₁ y⁽¹⁾₂ y⁽¹⁾₃ ∈ B⎫⎪⎪⎪

⎬⎪⎪⎪⎭ Set up the order for each pattern ̺ ∶{−,+}^Z2×2 →{1,2,... ,16} by

̺(y¹y2◇u1u2) = 8χ(y¹) + 4χ(y²) + 2χ(u¹) + χ(u²) + 1,

where χ ∶{−,+} → {0,1} is given by χ(−) = 0 and χ(+) = 1. This defines an ordering matrix whose elements consist of 4 × 2 patterns.

Let Y⁽¹⁾ and Y⁽²⁾ be the output space and hidden space extracted from the solution space Y. Define two labeling

L⁽¹⁾(y⁰y1y2y3◇u0u1u2u3) = u⁰u1u2u3

L⁽²⁾(y⁰y1y2y3◇u0u1u2u3) = y⁰y1y2y3

Then G⁽ⁱ⁾= (GT, L⁽ⁱ⁾) is the labeled graph representation of Y⁽ⁱ⁾for i= 1,2.

Theorem 4.1. Suppose Y⁽¹⁾ is the hidden space and Y⁽²⁾ is the output space. Then G⁽ⁱ⁾ is the labeled graph representation of Y⁽ⁱ⁾and Y⁽ⁱ⁾= XG⁽ⁱ⁾

for i= 1,2.

Define η ∶{−,+}^Z2×2 →{−,+}^Z2×2 and P4∈ R^16×16 by η(y¹y2◇u1u2) = u¹u2◇y1y2

P4(i,j) = { 1, χ(y¹y2◇u1u2) = i and χ ○ η(y¹y2◇u1u2) = j;

0, otherwise.

Let S⁽¹⁾ and S⁽²⁾ be the symbolic transition matrices of G⁽¹⁾ and G⁽²⁾, respectively. The following theorems are derived via an analogous method to the proof of the theorems in Section 2.3, and thus we omit the proof.

Theorem 4.2. If G⁽¹⁾and G⁽²⁾ are both right-resolving, then Y⁽¹⁾and Y⁽²⁾ are finitely equivalent. Moreover, Y⁽¹⁾≅ Y⁽²⁾ if and only if S⁽¹⁾P4= P⁴S⁽²⁾. Theorem 4.3. Suppose either G⁽¹⁾or G⁽²⁾is not right-resolving. Let(GY⁽¹⁾, L⁽¹⁾), (GY⁽²⁾, L⁽²⁾) be the minimal right-resolving graph representation of Y⁽¹⁾and Y⁽²⁾, respectively.

(a) There exists finite equivalence (W, ̃φY⁽¹⁾, ̃φ_Y(2)) between Y⁽¹⁾ and Y⁽²⁾. (b) If there exist E, F such that T_G

Y (1) = EF,T^G_{Y (}²) = FE, where T^G_{Y (}¹), T_G

Y (2)

is the transition matrices of G_Y(1) and G_Y(2), respectively. Then W⁽¹⁾≡ XG 1) is conjugate to W⁽²⁾≡ X^G 2).

(c) T_G

Y (1) ∼_FS T_G

Y (2) if and only if (△TG

Y (1), △⁺_T

GY (1), δ_TG

Y (1)) is isomor-phic to (△^TG

Y (2), △⁺_T

GY (2)

, δ_TG

Y (2)).

(d) If h(Y⁽¹⁾) = h(Y⁽²⁾), then there exists integral matrices E,F such that T_G

Y (1)E= ETG

Y (2) and T_G

Y (2)F = FTG

Y (1).

(e) If E is factor-like, then there exists a factor map π12 ∶ W⁽¹⁾ → W⁽²⁾, where and Similarly, if F is factor-like, there exists a factor map π21∶ W⁽²⁾→ W⁽¹⁾.

(f ) Suppose, for instance, there exists a factor map π12∶W⁽¹⁾→ W⁽²⁾. If Y⁽¹⁾ is a shift of finite type, then there exists π12∶Y⁽¹⁾→ Y⁽²⁾.

Suppose k ≥ 3. Without loss of generality, we assume that k = 2ℓ for some ℓ ∈ N. If k is odd, then we extend the size of the basic pattern to k +1 as above. Hence the basic set of admissible local patterns consists of patterns of size 2ℓ × 2. Set up the order for each pattern by

̺(y¹y2⋯y_2ℓ◇u1u2⋯u_2ℓ) = 1 +∑^2ℓ

i=1

(2^2ℓ−iχ(ui) + 2^4ℓ−iχ(yi))

Similar as above, let Y⁽¹⁾ and Y⁽²⁾ be the output space and hidden space extracted from the solution space Y. Define two labeling

L⁽¹⁾(y⁰y1⋯y_k−1◇u0u1⋯u_k−1) = u⁰u1⋯u_k−1 L⁽²⁾(y⁰y1⋯y_k−1◇u0u1⋯u_k−1) = y⁰y1⋯y_k−1

Then G⁽ⁱ⁾= (G^T, L⁽ⁱ⁾) is the labeled graph representation of Y⁽ⁱ⁾and Y⁽ⁱ⁾= X_G(i) for i= 1,2. Define η ∶ {−,+}^Zk×2 →{−,+}^Zk×2 and P_k∈ R^22k×22k by

η(y⁰y1⋯y_k−1◇u0u1⋯u_k−1) = u⁰u1⋯u_k−1◇y0y1⋯y_k−1 P_k(i,j) =⎧⎪⎪⎪

⎨⎪⎪⎪⎩

1, ̺(y⁰y1⋯y_k−1◇u0u1⋯u_k−1) = i

and ̺ ○ η(y⁰y1⋯y_k−1◇u0u1⋯u_k−1) = j;

0, otherwise.

Let S⁽¹⁾ and S⁽²⁾ be the symbolic transition matrices of G⁽¹⁾ and G⁽²⁾, respectively. We then have the following theorem.

Theorem 4.4. If G⁽¹⁾and G⁽²⁾ are both right-resolving, then Y⁽¹⁾and Y⁽²⁾ are finitely equivalent. Moreover, Y⁽¹⁾≅ Y⁽²⁾if and only if S⁽¹⁾P_k= P^kS⁽²⁾. The classification of Y⁽¹⁾and Y⁽²⁾is similar as that in Theorem 4.3. We omit therefore the description.

4.2 Multi-layer cellular neural networks

This section considers multi-layer cellular neural networks whose basic pat-terns are of size k × n. The foregoing elucidation infers that, without loss of generality, we may assume that k= 2ℓ for some ℓ ∈ N. Suppose the basic patterns are ordered by χ ∶{−,+}^Z2ℓ×n →{1,2,... ,2^2nℓ} and Pk is defined analogously as above. The solution space Y induces Y⁽ⁱ⁾ for i= 1,2,... ,n.

Similarly we have labeling L⁽ⁱ⁾ for i= 1,2,... ,n. This leads us to the fol-lowing theorem.

Proposition 4.5. Suppose S⁽ⁱ⁾ is the symbolic transition matrix of G⁽ⁱ⁾ = (G,L⁽ⁱ⁾) and G⁽ⁱ⁾ is right-resolving for i= 1,2,... ,n. Let

P_n,n−1;k= {(Pn−1,n−2⊗I2) ⋅ (C^p)1≤p≤2ⁿ⁻¹∶P_n−1,n−2∈ Pn−1,n−2;k, C_p is an ℓ × ℓ permutation matrix}

and

P_j,i;k= {Kj,i;k⁻¹ P_n,n−1;kK_j,i;k∶P_n,n−1;k∈ Pn,n−1;k}, 1 ≤ i < j ≤ n,(i,j) ≠ (n−1,n), where Kj,i;k is the permutation which bundles those vertices carrying the same label under L⁽ⁱ⁾ and L^(j) and P_2,1;k = {Pk}. Then Y⁽ⁱ⁾≅ Y^(j) if and only if S^(j)P_j,i;k = P^j,i;kS⁽ⁱ⁾ for some Pj,i;k∈ P^j,i;k, where 1≤ i < j ≤ n.

To discuss the relation between Y(i) and Y^(j) for i≠ j, we follow the flow chart as in Theorem 3.6 (See figure 11). Then, the main results follow.

Theorem 4.6. Suppose Y⁽¹⁾, Y⁽²⁾, . . . , Y⁽ⁿ⁾ are extracted from an n-layer cellular neural network with labeled graph representation G⁽¹⁾, G⁽²⁾, . . . , G⁽ⁿ⁾. If h(Y⁽¹⁾) = h(Y⁽²⁾) = ⋯ = h(Y⁽ⁿ⁾), then

(1) If G⁽ⁱ⁾ is right-resolving and Y⁽ⁱ⁾ is a shift of finite type for all i, then there exist factor maps π_ij∶Y⁽ⁱ⁾→ Y^(j) for 1≤ i,j ≤ n.

(2) If G⁽ⁱ⁾ is not right-resolving for some i, let G_Y(i) be a right-resolving labeled graph representation of Y⁽ⁱ⁾. Suppose Y⁽ⁱ⁾ is a shift of finite type for some i.

(i) If TG

Y (i) ∼_FSS T_G

Y (j) for some i, j, then Y⁽ⁱ⁾≅ Y^(j). (ii) If TG

Y (i) ∼_FS T_G

Y (j) for some i, j, then there exists a factor map between Y⁽ⁱ⁾ and Y^(j) provided there is a factor-like matrix F which commutes with TG

Y (i) and TG

Y (j).

(iii) Otherwise, Y⁽ⁱ⁾ is strictly finitely equivalent to Y^(j).

5 Discussion

This elucidation investigates the relations between subspaces of the solution space of a multi-layer cellular neural network. A small modification of the above procedure allows for decoupling the solution space of an n-layer cel-lular neural network into arbitrary k subspaces for 2≤ k ≤ n. The existence of a factor map between two subspaces depends on whether there exists a factor map between their covering spaces. Note that a covering space of a sofic shift is a shift of finite type. In other words, to classify the subspaces of a solution space is equivalent to the classification of subshifts of finite type induced by multi-layer cellular neural networks. It is known that shift equivalence cannot conclusively establish the conjugacy between two arbi-trary subshifts of finite type [18, 17]. We conjecture that shift equivalence implies that two subshifts of finite type induced from a multi-layer cellular neural network are conjugate.

Conjecture 1. Suppose Y is the solution space of a multi-layer neural net-work. Let Y⁽¹⁾, Y⁽²⁾be two subshifts of finite type such that(Y,φ⁽¹⁾, φ⁽²⁾) is a finite equivalence between Y⁽¹⁾ and Y⁽²⁾. Then Y⁽¹⁾ is conjugate to Y⁽²⁾ if and only if Y⁽¹⁾and Y⁽²⁾ are shift equivalent.

References

[1] P. Arena, S. Baglio, L. Fortuna, and G. Manganaro. Self-organization in a two-layer CNN. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., 45:157–162, 1998.

[2] J.-C. Ban and C.-H. Chang. On the monotonicity of entropy for multi-layer cellular neural networks. Internat. J. Bifur. Chaos Appl. Sci.

Engrg., 19:in press, 2009.

[3] J.-C. Ban and S.-S. Lin. Patterns generation and transition matrices in multi-dimensional lattice models. Discrete Contin. Dyn. Syst., 13:

637–658, 2005.

[4] J.-C. Ban, C.-H. Chang, and S.-S. Lin. Diamond in multi-layer cellular neural networks. submitted.

[5] J.-C. Ban, C.-H. Chang, S.-S. Lin, and Y.-H Lin. Spatial complicity in multi-layer cellular neural networks. J. Differential Equations, 246:

552–580, 2009.

[6] L. O. Chua and T. Roska. Cellular neural networks and visual comput-ing. Cambridge University Press, 2002.

[7] L. O. Chua and L. Yang. Cellular neural networks: Theory. IEEE Trans. Circuits Syst., 35:1257–1272, 1988.

[8] L. O. Chua and L. Yang. Cellular neural networks: Applications. IEEE Trans. Circuits Syst., 35:1273–1290, 1988.

[9] K. R. Crounse and L. O. Chua. Methods for image processing and pattern formation in cellular neural networks: A tutorial. IEEE Trans.

Circuits Syst., 42:583–601, 1995.

[10] K. R. Crounse, T. Roska, and L. O. Chua. Image halftoning with cellular neural networks. IEEE Trans. Circuits Syst., 40:267–283, 1993.

[11] Marco Gilli, Mario Biey, and Paolo Checco. Equilibrium analysis of cellular neural networks. IEEE Trans. Circuits Syst. I Regul. Pap., 51:

903–912, 2004.

[12] C.-H. Hsu, J. Juang, S.-S. Lin, and W.-W. Lin. Cellular neural net-works: Local patterns for general template. Internat. J. Bifur. Chaos Appl. Sci. Engrg., 10:1645–1659, 2000.

[13] Makoto Itoh and Leon O. Chua. Structurally stable two-cell cellular neural networks. Internat. J. Bifur. Chaos Appl. Sci. Engrg., 14:2579–

2653, 2004.

[14] J. Juang and S.-S. Lin. Cellular neural networks: Mosaic pattern and spatial chaos. SIAM J. Appl. Math., 60:891–915, 2000.

[15] Yun-Quan Ke and Chun-Fang Miao. Existence analysis of stationary solutions for rtd-based cellular neural networks. Internat. J. Bifur.

Chaos Appl. Sci. Engrg., 20:2123–2136, 2010.

[16] Yunquan Ke. The mosaic patterns of cnn with symmetric feedback template. Internat. J. Bifur. Chaos Appl. Sci. Engrg., 18:375–390, 2008.

[17] K. H. Kim and F. W. Roush. Williams’s conjecture is false for reducible subshifts. J. Amer. Math. Soc., 5:213–215, 1992.

[18] K. H. Kim and F. W. Roush. The williams conjecture is false for irreducible subshifts. Ann. of Math., 149:545–558, 1999.

[19] B. Kitchens. Symbolic dynamics. One-sided, two-sided and countable state Markov shifts. Springer-Verlag, New York, 1998.

[20] Guodong Li, Lequan Min, and Hongyan Zang. Color edge detections based on cellular neural network. Internat. J. Bifur. Chaos Appl. Sci.

Engrg., 18:1231–1242, 2008.

[21] Xue Mei Li and Li Hong Huang. On the complete stability of cellular neural networks with external inputs and bias. Acta Math. Appl. Sin., 26:475–486, 2003.

[22] Xuemei Li. Analysis of complete stability for discrete-time cellular neu-ral networks with piecewise linear output functions. Neuneu-ral Comput., 21:1434–1458, 2009.

[23] Xuemei Li and Zhaohui Yuan. Existence of periodic solutions and closed invariant curves in a class of discrete-time cellular neural networks.

Phys. D, 238:1658–1667, 2009.

[24] S.-S. Lin and C.-W. Shih. Complete stability for standard cellular neu-ral networks. Internat. J. Bifur. Chaos Appl. Sci. Engrg., 9:909–918, 1999.

[25] D. Lind and B. Marcus. An introduction to symbolic dynamics and coding. Cambridge University Press, Cambridge, 1995.

[26] V. Murugesh. Image processing applications via time-multiplexing cel-lular neural network simulator with numerical integration algorithms.

Int. J. Comput. Math., 87:840–848, 2010.

[27] Taisuke Nishio and Yoshifumi Nishio. Periodic pattern formation and its applications in cellular neural networks. IEEE Trans. Circuits Syst.

I. Regul. Pap., 55:2736–2742, 2008.

[28] Jun Peng, Du Zhang, and Xiaofeng Liao. A digital image encryption al-gorithm based on hyper-chaotic cellular neural network. Fund. Inform., 90:269–282, 2009.

[29] C.-W. Shih. Influence of boundary conditions on pattern formation and spatial chaos in lattice systems. SIAM J. Appl. Math., 61:335–368, 2000.

[30] N. Takahashi, K. Ishitobi, and T. Nishi. Sufficient conditions for one-dimensional cellular neural networks to perform connected component detection. Nonlinear Anal. Real World Appl., 11:4202–4213, 2010.

[31] Xavier Vilas´ıs-Cardona and Mireia Vinyoles-Serra. Two neuron cnns:

search for limit cycles. Internat. J. Bifur. Chaos Appl. Sci. Engrg., 20:

1137–1173, 2010.

[32] R. F. Williams. Classification of subshifts of finite type. Ann. of Math., 98:120–153, 1973.

[33] R. F. Williams. Classification of subshifts of finite type, errata. Ann.

of Math., 99:380–381, 1974.

[34] S. Xavier-de Souza, M.E. Yalcin, J.A.K. Suykens, and J. Vandewalle.

Toward CNN chip-specific robustness. IEEE Trans. Circuits Syst. I, Reg. Papers, 51:892 – 902, 2004.

[35] Z. Yang, Y. Nishio, and A. Ushida. Image processing of two-layer CNNs

−applications and their stability. IEICE Trans. Fundam., E85-A:2052–

2060, 2002.

在文檔中 On The Structure of Multi-Layer Cellular Neural Networks (頁 33-41)