Structures in multi-layer cellular neural networks

The propositions and theorems we obtain in previous sections still hold for general multi-layer neural networks. In this section, we only state the results instead of giving their explicit proofs.

5.1. Two-layer cellular neural networks

First we consider a two-layer cellular neural network whose basic set of admissible local patterns are of size k

×

^{2, k}



3. For simplicity, we assume that each parameter is nonzero.

Suppose that k

=

3, we embed our problem in the case that k

=

4. In other words, suppose

B ⊆

Let Y⁽¹⁾and Y⁽²⁾be the output space and hidden space extracted from the solution space Y. Define two labeling theorems are derived via an analogous method to the proof of the theorems in Section 3.3, and thus we omit the proof.

Theorem 5.2. If

G

^(1)and

G

⁽²⁾are both right-resolving, then Y⁽¹⁾and Y⁽²⁾are finitely equivalent. Moreover, Y⁽¹⁾

∼ =

^Y(2)if and only if S⁽¹⁾P4

=

P4S⁽²⁾.

Theorem 5.3. Suppose either

G

^(1)or

G

⁽²⁾is not right-resolving. Let

(

G_Y(1)

, L

⁽¹⁾

)

,

(

G_Y(2)

, L

⁽²⁾

)

be the minimal right-resolving graph representation of Y⁽¹⁾and Y⁽²⁾, respectively.

(a) There exists finite equivalence

(

W

,  φ

_Y(1)

,  φ

_Y(2)

)

between Y⁽¹⁾and Y⁽²⁾.

(e) If E is factor-like, then there exists a factor map

π

12

:

^W(1)

→

^W(2), where and similarly, if F is factor-like, there exists a factor map

π

21

:

^W(2)

→

^W(1).

(f) Suppose, for instance, there exists a factor map

π

12

:

^W(1)

→

^{W(2). If Y(1)}is a shift of finite type, then there exists

π ¯

12

:

^Y(1)

→

^Y(2).

Suppose k



3. Without loss of generality, we assume that k

=

²



for some

 ∈ N

. If k is odd, then we extend the size of the basic pattern to k

+

1 as above. Hence the basic set of admissible local patterns consists of patterns of size 2

 ×

2. Set up the order for each pattern by

 (

y₁y₂

· · ·

^y2



^u1u₂

· · ·

^u2

) =

¹

+

2



i=¹



2^2−i

χ (

u_i

) +

^24−i

χ (

y_i

)  .

Similar as above, let Y⁽¹⁾and Y⁽²⁾be the output space and hidden space extracted from the solution space Y. Define two labeling

L

⁽¹⁾

(

y0y1

· · ·

yk−¹



u0u1

· · ·

uk−¹

) =

u0u1

· · ·

uk−¹

, L

⁽²⁾

(

y0y1

· · ·

yk−¹



u0u1

· · ·

uk−¹

) =

y0y1

· · ·

yk−¹

.

Then

G

⁽ⁱ⁾

= (

^GT

, L

⁽ⁱ⁾

)

is the labeled graph representation of Y⁽ⁱ⁾ and Y⁽ⁱ⁾

=

^X_G^{(i) for i}

=

¹

,

2. Define

¯

η : {−, +}

^Zk×2

→ {−, +}

^{Zk×2 and P}k

∈ R

^{22k×22k by}

¯

η (

y0y1

· · ·

y_k−1



u0u1

· · ·

u_k−1

) =

u0u1

· · ·

u_k−1



y0y1

· · ·

y_k−1

,

P_k

(

i

,

j

) =

⎧ ⎨

⎩

1

,  (

y₀y₁

· · ·

^yk−¹



^u0u₁

· · ·

^uk−¹

) =

^{i and}

 _{◦ ¯} η (

y0y1

· · ·

^yk−1



^u0u1

· · ·

^uk−1

) =

^j;

0

,

otherwise.

Let S⁽¹⁾and S⁽²⁾be the symbolic transition matrices of

G

^(1)and

G

⁽²⁾, respectively. We then have the following theorem.

Theorem 5.4. If

G

^(1)and

G

⁽²⁾are both right-resolving, then Y⁽¹⁾and Y⁽²⁾are finitely equivalent. Moreover, Y⁽¹⁾

∼ =

^Y(2)if and only if S⁽¹⁾Pk

=

PkS⁽²⁾.

The classification of Y⁽¹⁾ and Y⁽²⁾ is similar as that in Theorem 5.3. We omit therefore the de-scription.

5.2. Multi-layer cellular neural networks

This section considers multi-layer cellular neural networks whose basic patterns are of size k

×

^n.

The foregoing elucidation infers that, without loss of generality, we may assume that k

=

²



for some

 ∈ N

. Suppose the basic patterns are ordered by

χ _{: {−, +}}

^Z2×n

_{→ {}

1

,

2

, . . . ,

2^2n

}

^{and P}kis defined analogously as above. The solution space Y induces Y⁽ⁱ⁾for i

=

¹

,

2

, . . . ,

n. Similarly, we have labeling

L

^{(i)for i}

=

¹

,

2

, . . . ,

n. This leads us to the following theorem.

Proposition 5.5. Suppose S⁽ⁱ⁾is the symbolic transition matrix of

G

⁽ⁱ⁾

= (

^G

, L

⁽ⁱ⁾

)

and

G

⁽ⁱ⁾is right-resolving for i

=

¹

,

2

, . . . ,

n. Let

P

n,n−¹;^k

= 

(

P_n−1,n−2

⊗

^I2

) · (

^Cp

)

_1p2n−1: P_n−1,n−2

∈ P

n−¹,n−²;^k

,

Cpis an

 × 

permutation matrix

and

P

j,i;^k

= 

K⁻_j,¹_i;kP_n,n−1;kK_j,i;k: P_n,n−1;k

∈ P

n,n−¹;^k

,

1



ⁱ

<

j



ⁿ

, (

i

,

j

) = (

ⁿ

−

¹

,

n

),

where Kj,i;^kis the permutation which bundles those vertices carrying the same label under

L

^(i)and

L

^(j) and

P

2,1;^k

= {

^Pk

}

^{. Then Y(i)}

∼ =

^Y(j) if and only if S^(j)P_j,i;k

=

^Pj,i;^kS⁽ⁱ⁾ for some P_j,i;k

∈ P

j,i;^k, where 1



ⁱ

<

j



^n.

To discuss the relation between Y

(

i

)

and Y^(j)for i

=

j, we follow the flow chart as in Theorem 4.6 (see Fig. 11). Then, the main results follow.

Theorem 5.6. Suppose Y⁽¹⁾

,

Y⁽²⁾

, . . . ,

Y⁽ⁿ⁾are extracted from an n-layer cellular neural network with labeled graph representation

G

⁽¹⁾

, G

⁽²⁾

, . . . , G

^{(n). If h}

(

Y⁽¹⁾

) =

^h

(

Y⁽²⁾

) = · · · =

^h

(

Y⁽ⁿ⁾

)

, then

1. If

G

⁽ⁱ⁾is right-resolving and Y⁽ⁱ⁾is a shift of finite type for all i, then there exist factor maps

π ¯

i j

:

^Y(i)

→

Y^(j)for 1



ⁱ

,

j



^n.

2. If

G

⁽ⁱ⁾is not right-resolving for some i, let

G

Y⁽ⁱ⁾ be a right-resolving labeled graph representation of Y⁽ⁱ⁾. Suppose Y⁽ⁱ⁾is a shift of finite type for some i.

(i) If TG_{Y (i)}

∼

_FS STG_{Y ( j)}for some i

,

j, then Y⁽ⁱ⁾

∼ =

^Y(j).

(ii) If TG_{Y (i)}

∼

FSTG_{Y ( j)}for some i

,

j, then there exists a factor map between Y⁽ⁱ⁾and Y^(j)provided there is a factor-like matrix F which commutes with TG_{Y (i)}and TG_{Y ( j)}.

(iii) Otherwise, Y⁽ⁱ⁾is strictly finitely equivalent to Y^(j).

6. Discussion

This elucidation investigates the relations between subspaces of the solution space of a multi-layer cellular neural network. A small modification of the above procedure allows for decoupling the solu-tion space of an n-layer cellular neural network into arbitrary k subspaces for 2



^k



n. The existence of a factor map between two subspaces depends on whether there exists a factor map between their covering spaces. Note that a covering space of a sofic shift is a shift of finite type. In other words, to classify the subspaces of a solution space is equivalent to the classification of subshifts of finite type induced by multi-layer cellular neural networks. It is known that shift equivalence cannot con-clusively establish the conjugacy between two arbitrary subshifts of finite type [35,36]. We conjecture that shift equivalence implies that two subshifts of finite type induced from a multi-layer cellular neural network are conjugate.

Conjecture 1. Suppose Y is the solution space of a multi-layer neural network. Let Y⁽¹⁾

,

Y⁽²⁾be two subshifts of finite type such that

(

Y

, φ

⁽¹⁾

, φ

⁽²⁾

)

is a finite equivalence between Y⁽¹⁾and Y⁽²⁾. Then Y⁽¹⁾is conjugate to Y⁽²⁾if and only if Y⁽¹⁾and Y⁽²⁾are shift equivalent.

Acknowledgments

The authors wish to express their gratitude to the anonymous referees. Their comments make an improvement to this paper.

References

[1] P. Arena, S. Baglio, L. Fortuna, G. Manganaro, Self-organization in a two-layer CNN, IEEE Trans. Circuits Syst. I. Fundam.

Theory Appl. 45 (1998) 157–162.

[2] L.O. Chua, L. Yang, Cellular neural networks: Theory, IEEE Trans. Circuits Syst. 35 (1988) 1257–1272.

[3] L.O. Chua, L. Yang, Cellular neural networks: Applications, IEEE Trans. Circuits Syst. 35 (1988) 1273–1290.

[4] L.O. Chua, T. Roska, Cellular Neural Networks and Visual Computing, Cambridge University Press, 2002.

[5] K.R. Crounse, L.O. Chua, Methods for image processing and pattern formation in cellular neural networks: A tutorial, IEEE Trans. Circuits Syst. 42 (1995) 583–601.

[6] K.R. Crounse, T. Roska, L.O. Chua, Image halftoning with cellular neural networks, IEEE Trans. Circuits Syst. 40 (1993) 267–

283.

[7] S. Xavier-de Souza, M. Yalcin, J. Suykens, J. Vandewalle, Toward CNN chip-specific robustness, IEEE Trans. Circuits Syst. I.

Regul. Pap. 51 (2004) 892–902.

[8] Z. Yang, Y. Nishio, A. Ushida, Image processing of two-layer CNNs – Applications and their stability, IEICE Trans. Fun-dam. E85-A (2002) 2052–2060.

[9] G. Li, L. Min, H. Zang, Color edge detections based on cellular neural network, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 18 (2008) 1231–1242.

[10] V. Murugesh, Image processing applications via time-multiplexing cellular neural network simulator with numerical inte-gration algorithms, Int. J. Comput. Math. 87 (2010) 840–848.

[11] J. Peng, D. Zhang, X. Liao, A digital image encryption algorithm based on hyper-chaotic cellular neural network, Fund.

Inform. 90 (2009) 269–282.

[12] S.-N. Chow, J. Mallet-Paret, E.V. Vleck, Pattern formation and spatial chaos in spatially discrete evolution equations, Random Comput. Dynam. 4 (1996) 109–178.

[13] S.-N. Chow, W. Shen, Dynamics in a discrete Nagumo equation spatial topological chaos, SIAM J. Appl. Math. 55 (1995) 1764–1781.

[14] J. Juang, S.-S. Lin, Cellular neural networks: Mosaic pattern and spatial chaos, SIAM J. Appl. Math. 60 (2000) 891–915.

[15] S.-S. Lin, C.-W. Shih, Complete stability for standard cellular neural networks, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 9 (1999) 909–918.

[16] Y. Ke, The mosaic patterns of CNN with symmetric feedback template, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 18 (2008) 375–390.

[17] Y.-Q. Ke, C.-F. Miao, Existence analysis of stationary solutions for RTD-based cellular neural networks, Internat. J. Bifur.

Chaos Appl. Sci. Engrg. 20 (2010) 2123–2136.

[18] X. Li, Analysis of complete stability for discrete-time cellular neural networks with piecewise linear output functions, Neural Comput. 21 (2009) 1434–1458.

[19] C.-W. Shih, Influence of boundary conditions on pattern formation and spatial chaos in lattice systems, SIAM J. Appl.

Math. 61 (2000) 335–368.

[20] M. Itoh, L.O. Chua, Structurally stable two-cell cellular neural networks, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 14 (2004) 2579–2653.

[21] M. Gilli, M. Biey, P. Checco, Equilibrium analysis of cellular neural networks, IEEE Trans. Circuits Syst. I. Regul. Pap. 51 (2004) 903–912.

[22] X.M. Li, L.H. Huang, On the complete stability of cellular neural networks with external inputs and bias, Acta Math. Appl.

Sin. 26 (2003) 475–486.

[23] X. Li, Z. Yuan, Existence of periodic solutions and closed invariant curves in a class of discrete-time cellular neural net-works, Phys. D 238 (2009) 1658–1667.

[24] T. Nishio, Y. Nishio, Periodic pattern formation and its applications in cellular neural networks, IEEE Trans. Circuits Syst. I.

Regul. Pap. 55 (2008) 2736–2742.

[25] N. Takahashi, K. Ishitobi, T. Nishi, Sufficient conditions for one-dimensional cellular neural networks to perform connected component detection, Nonlinear Anal. Real World Appl. 11 (2010) 4202–4213.

[26] X. Vilasís-Cardona, M. Vinyoles-Serra, Two neuron CNNs: Search for limit cycles, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 20 (2010) 1137–1173.

[27] J.-C. Ban, C.-H. Chang, S.-S. Lin, Y.-H. Lin, Spatial complexity in multi-layer cellular neural networks, J. Differential Equa-tions 246 (2009) 552–580.

[28] C.-H. Hsu, J. Juang, S.-S. Lin, W.-W. Lin, Cellular neural networks: Local patterns for general template, Internat. J. Bifur.

Chaos Appl. Sci. Engrg. 10 (2000) 1645–1659.

[29] J.-C. Ban, C.-H. Chang, On the monotonicity of entropy for multi-layer cellular neural networks, Internat. J. Bifur. Chaos Appl. Sci. Engrg. 19 (2009) 3657–3670.

[30] B. Kitchens, Symbolic Dynamics. One-Sided, Two-Sided and Countable State Markov Shifts, Springer-Verlag, New York, 1998.

[31] D. Lind, B. Marcus, An Introduction to Symbolic Dynamics and Coding, Cambridge University Press, Cambridge, 1995.

[32] J.-C. Ban, S.-S. Lin, Patterns generation and transition matrices in multi-dimensional lattice models, Discrete Contin. Dyn.

Syst. 13 (2005) 637–658.

[33] R.F. Williams, Classification of subshifts of finite type, Ann. of Math. 99 (1974) 380–381 (Erratum).

[34] R.F. Williams, Classification of subshifts of finite type, Ann. of Math. 98 (1973) 120–153.

[35] K.H. Kim, F.W. Roush, The Williams conjecture is false for irreducible subshifts, Ann. of Math. 149 (1999) 545–558.

[36] K.H. Kim, F.W. Roush, Williams’s conjecture is false for reducible subshifts, J. Amer. Math. Soc. 5 (1992) 213–215.

[37] J.-C. Ban, C.-H. Chang, S.-S. Lin, Diamond in multi-layer cellular neural networks, submitted for publication, 2011.

在文檔中 On the structure of multi-layer cellular neural networks (頁 31-35)