Architectural Design and Analysis of Learnable
Self-Feedback Ratio-Memory Cellular Nonlinear
Network (SRMCNN) for Nanoelectronic Systems
Jui-Lin Lai, Member, IEEE, and Peter Chung-Yu Wu, Fellow, IEEE
Abstract—In this paper, a learnable cellular nonlinear network (CNN) with space-variant templates, ratio memory (RM), and modified Hebbian learning algorithm is proposed and analyzed. By integrating both the modified Hebbian learning algorithm with the self-feedback function and a ratio memory into CNN architecture, the resultant ratio-memory (RMCNN) is called the self-feedback RMCNN (SRMCNN) which can serve as the associative memory. It can generate the absolute weights and then transform them into the ratioed A-template weights as the ratio memories for recognizing noisy input patterns. Simulation results have shown that with the stronger feature enhancement effect, the SRMCNN under constant leakage current can store and recognize more patterns than the RMCNN. For 18 18 SRMCNN, 93 noisy patterns with a uniform distribution noise level of 0.8 and a variance of normal distribution noise of 0.3 can be learned, stored, and recognized with 100% success rate. The SRMCNN has greater learning and recognition capability when the learned patterns are simpler and the noise is lower. For the learning and recognition of complicated patterns, the allowable pattern number is decreased for a 100% success rate. Simulation results have successfully verified the correct functions and better performance of SRMCNN in the pattern recognition. With high integration capability and excellent pattern association perfor-mance, the proposed SRMCNN can be applied to nanoelectronic associative-memory systems for image processing applications.
Index Terms—Cellular nonlinear network, modified Hebbian learning algorithm, nanoelectronic, ratio memory, template.
I. INTRODUCTION
A
S INTRODUCED by Chua and Yang [1], [2], cellular non-linear network (CNN) with locally connected neighboring cells have the inherent advantage of being easily implemented in VLSI for various applications. Many image operations in CNN with suitable templates have been successfully explored [3], [4] and realized in many applications. Moreover, the CNN can be used to classify and recognize image patterns through appro-priate learning algorithms. Recently, this innovative application of CNN has attracted more research effort. Some important re-sults have been reported in the literature [5]–[13].Manuscript received December 19, 2002; revised February 27, 2004. This work was supported by the National Science Council of Taiwan, R.O.C., under Contract NSC90-2215-E-009-113.
J.-L. Lai is with the Integrated Circuits and Systems Laboratory, Depart-ment of Electronics Engineering, National Chiao-Tung University, Hsinchu City 30050, Taiwan, R.O.C. He is also with the Mixed-Signal Integration Circuit Laboratory, Department of Electronics Engineering, National United Univer-sity, Miao-Li 360, Taiwan, R.O.C. (e-mail: [email protected]).
P. C.-Y. Wu is with the Integrated Circuits and Systems Laboratory, Depart-ment of Electronics Engineering, National Chiao-Tung University, Hsinchu City 30050, Taiwan, R.O.C. (e-mail: [email protected]).
Digital Object Identifier 10.1109/TVLSI.2004.836309
The Hebbian learning algorithm can be used to perform unsupervised learning operations in a neural network system, in which the learned pattern signal is equal to the neuron’s output. One Hebbian learning algorithm, called the discrete Hebbian learning algorithm, has been incorporated into CNN with some modification terms to generate associative memories for the learning and recognizing of image patterns [12], [13]. Modified Hebbian learning is used to implement the 18 18 CNN for pattern learning over a fixed period. The ratio memory (RM) in the Grossberg out-star structure is also used to form the template coefficients in the CNN for image recognition. The resultant structure is called the ratio-memory CNN (RMCNN) [12], [13]. To determine the four coefficients of the template but not self-feedback coefficient for the cell of the proposed RMCNN, the pixel values of the nearest four neigh-boring cells are multiplied by the pixel value of cell , and the products are summed for all input patterns. Then, the accumulated product is transformed into a ratio to form the coefficient of the A template. The proposed RMCNN can learn and recognize three (five) patterns in the 9 9 (18 18) neuron array. The structure of RMCNN has been implemented in CMOS technology and its function has been successfully verified [12], [13].
The modified Hebbian learning algorithm used in the RMCNN can be modified to include a self-feedback term [14]. The modified algorithm is called the modified Hebbian learning algorithm with self-feedback. In this paper, the RMCNN with the modified Hebbian learning algorithm with self-feed-back is proposed and analyzed. The new RMCNN is called self-feedback RMCNN (SRMCNN). In the learning process of the proposed SRMCNN, the features from input exemplar patterns are considered to update the weights. The operation of SRMCNN retains the feature enhancement effect of the RM. Detailed analysis and simulation results has shown that the SRMCNN can recognize up to 93 noisy patterns with a 100% success rate and 98 noisy patterns with a 97% success rate after learning the input exemplar patterns in uniform (normal) noise level is 0.8 (0.3). Thus, the capacity for learning and recognizing patterns is greatly improved.
The paper is organized as follows. In Section II, the oper-ational principles, the modified Hebbian learning algorithm with self-feedback, and the embedded ratio memory in the SRMCNN are presented. Section III describes the architecture of the SRMCNN. In Section IV, the simulation results of SRMCNN are demonstrated and analyzed. Some phenomena are also discussed. Finally, conclusions are drawn.
LAI AND WU: ARCHITECTURAL DESIGN AND ANALYSIS 1183
Fig. 1. General architecture of the SRMCNN.
II. OPERATIONALPRINCIPLE ANDLEARNINGALGORITHM
In a CNN, the behavior of a regular cell and its neigh-boring cells can be expressed by the differential state equation, in terms of their input, state, and output variables as [1]–[4]
(1)
and the equation of the cell output is [1]–[4] if
if if
(2)
where represents the cell state, is the cell output from cell in the r-neighborhood system of the cell , is the cell input from cell in , is the threshold of cell , is bipolar activation function, and is the weight of template that correlates to . In an M N CNN cell array, the r-neigh-borhood system of cell is defined as a set of cells that includes cell and its neighborhood cells. The term r is an integer that represents the number of the neighbor-hood layers. can be expressed by the following:
(3) The general architecture of the SRMCNN is depicted in Fig. 1 [12], [13] where the RM is used to realize the -tem-plate weights of two neighboring cells, and SRM is used to realize the self-feedback weight of the cell. In the SRMCNN, a coupled template, an uncoupled template, and neighborhood is adopted. The space-variant template has a
self-feedback coefficient and four nearest neighboring coeffi-cients. The template has only one coefficient that corresponds to the input of cell . Both and templates of can be expressed as
for (4) The outermost boundary cells are called the edge cells. They are commonly used to realize fixed (Dirichlet) boundary conditions. The output and input of those boundary cells are set to zero.
The modified Hebbian learning algorithm with self-feedback is applied in the SRMCNN to determine the updated volume of the weight vector at as
(5) (6) where m is the number of learning patterns, is the pixel value of the th row, and the th column in the th pattern of m learned patterns with the value 1 or 1, is the pixel of the cell
of neighboring cells including cell ,
in (5) is the weight associated with cell and its neigh-boring cells , and in (6) is the threshold of cell , which is set to zero. Note that the self-feedback terms
is defined in (5).
In the learning period, the weights are generated in parallel from and accumulated for all m learned exemplar patterns. They are updated simultaneously when an exemplar pattern is input at a given time. Then its magnitude
is stored on the capacitor to generate the ratio weights. Ac-cording to (5), if the product of is positive, the weight of template is increased. Otherwise, is de-creased. Since the self-feedback term is always positive, the self-feedback weight is one of the largest weights among the five weights in (5).
In the elapsed period, starting from , the leakage cur-rent associated with capacitor gradually decreases the stored voltage as time elapses. Since the leakage current is almost constant, the change of on capacitor
can be written as
(7) The RM is used to generate the ratio weight of the template in the recognition period. The noisy patterns are input to the SRMCNN with the ratio weights to perform the recog-nition operation. The derivative of the cell state is ex-pressed as
Fig. 2. Detailed architecture of (a) two neighboring cells and their ratio memories (RM) and (b) the S block in the SRMCNN during the learning period.
and the ratio weights are generated according to [15]
(9)
The ratio weight in (9) has the effect of feature en-hancement. When the weight magnitude exceeds the mean value of all terms of the cell , it is increased gradually with time. Otherwise, the weight decreases gradually. Since the self-feedback weight or is one of the largest weights in the cell , the corresponding self-feedback ratio weight is the largest in the template. With the fea-tures of patterns can be enhanced to reject the noise. Thus, the capability of recognition for noisy patterns is significantly im-proved by the SRMCNN.
In the recognition period, the outputs are adjusted according to (8) for noisy input patterns with either uniform or Gaussian (normal) noise distribution. The output pattern noise is gradu-ally eliminated through a feedback-type interaction. The out-puts of all neurons are adjusted to eliminate noise during the recognition period until no further change is detected. Finally, the SRMCNN reaches its stable state.
The energy function of a CNN in quadratic form [16] can be expressed as
(10) When all the cells become saturated in the recognition period, we have and the SRMCNN results in a stable output
with the energy function converged to its local minimum. The minima of E correspond to stable states. The final recognized pattern represents one local minimum of the energy function.
III. SRMCNN ARCHITECTURE
The detailed block diagram of two neighboring CNN cells and their RM in the SRMCNN are shown in Fig. 2(a), and the de-tailed block diagram of the S block is shown Fig. 2(b) when the SRMCNN is operated during the learning period. In Fig. 2(a), the block T1 is a V–I converter used to convert the voltage of input patterns into current. The block T2d is a V–I converter with a one-half absolute-value circuit and a sign-detection circuit to generate the absolute value of output current and detect the sign of the cell state , respectively. The CNN cell is formed by T1, T2d, , and as indicated in Fig. 2(a) [13]. The block M/D [13] in Fig. 2(a) is a combined four-quad-rant multiplier and a two-quadfour-quad-rant divider circuit. The block is used to realize the modified Hebbian learning algorithm with self-feedback during the learning period. It is also used to mul-tiply perform the multiplication and in the recog-nition period. The resultant absolute weight during the learning period is stored in the capacitor in the S block of Fig. 2(b). In Fig. 2(b), the block T2L transfers the absolute value of the voltage stored in to and stores its sign in the latch circuit. The resistor in parallel with is used to gener-ated the absolute voltage from the output current of block T2L and to store the voltage on . Block T3 is also a V-I converter to convert the voltage of into current. The output current of T3 is sent to the sum block and summed with the currents from
LAI AND WU: ARCHITECTURAL DESIGN AND ANALYSIS 1185
Fig. 3. Detailed architecture of (a) two neighboring cells and their ratio memories (RM) and (b) the S block in the SRMCNN during the elapsed period.
neighboring cells. The summed current is sent to the M/D block to generate ratio-memory. Both M/D and S blocks form the RM among CNN cells as indicated in Fig. 2(a).
In Fig. 2(a), the exemplar patterns are input in order read into the cell and the input voltage of the th input pattern is sent to T1 to be converted into current and then to T2d to extract its absolute current value and sign. Then the converted absolute currents and from two neigh-boring cells are sent to the four-quadrant multiplier in the M/D block to generate the product. The generated product in the cur-rent mode charges the capacitor for the period to gen-erate the voltage on . This operation is repeated for pat-terns to sum the voltages of . Finally, the weight voltage stored on at when the learning period ends, can be written as
(11) where is the current of the th input patterns sent to the cell , is the current of the th pattern sent to the cell of neighboring cells, Ib is a constant bias current, is the weight voltage stored on at , and is the learning time of each input pattern. Through T2L, the absolute value of the weight is stored on the capacitor , whereas the sign of is stored in the latch circuit of T2L.
In Fig. 2(a), the voltage weight is di-rectly generated by the current product of changing on
the capacitor for the period . is stored on the capacitor . Then, the absolute value of is trans-ferred and stored on the capacitor .
In the elapsed period, the configuration of SRMCNN is shown in Fig. 3(a), where is disconnected from the block T2L as shown in Fig. 3(b). The leakage current associ-ated with gradually decreases of .
In the recognition period, the configuration of the SRMCNN is shown in Fig. 4, and that of S block are the same as that in the elapsed period. The voltage of the test pattern to be recognized is input to T1 and converted into the current . The absolute weight voltage stored on is converted into the current through T3 and summed with the currents from other neighboring cells. The summed current, the weight current , and the cell output current are sent to the M/D block to yield the current that corresponds to the term in (1), which is then summed with the currents from other neighboring cells, the input current , and the threshold current to generate the cell state current . The current is converted into the voltage through resistor . Thus,
can be expressed as
(12) where is the empirical gain. Ideally . The ratioed weight in (12) is
Fig. 4. Detailed architecture of two neighboring cells and their ratio memories (RM) in the SRMCNN during the recognition period.
generated by the two-quadrant divider in the M/D block with its sign equal to the sign of latched in T2L, whereas the is multiplied by the ratioed weight by the four-quadrant multiplier of M/D using the latched sign of and the sign of in T2d. The current of input patterns is summed with the five weighted outputs from neighboring cells during the recognition period and converted into a voltage through the resistor and the parasitic capacitor to form the cell state
.
The generated is sent to T2d to generate the cur-rent and . The block T2d realizes by separating its magnitude and sign. The sign is detected in the block T2d and the voltage is .
In the proposed SRMCNN, each cell requires an extra feedback ratio-memory with M/D and S block to realize the self-feedback weight or . As in the original RMCNN, eight sets of M/D and S block are required to generate and store the ratio weight from cell and the ratio weight from cell , respectively. Thus, five sets of M/D and S block per cell are required in the architecture of SRMCNN. As compared with RMCNN, the increased hardware is small but the performance in pattern recognition is greatly improved.
The SRMCNN also can be integrated into the conventional CNNUM, and is called SRMCNNUM. The chip area of the cell, the core cell array, and the SRMCNNUM are estimated by using different CMOS process technologies. Table I lists these areas.
IV. SIMULATIONRESULTS
Matlab software is used to simulate the operations of the pro-posed SRMCNN with 18 18 neurons, the direct neighbor-hood , and the modified Hebbian learning algorithm with self-feedback. The 18 18 SRMCNN can process pat-terns with 324 pixels. In each pattern, a black pixel is expressed by 1 whereas a white pixel by 1. To elucidate the effect of
TABLE I
ESTIMATEDCHIPAREAS OFCELL, CORECELLARRAY,ANDSRMCNNUM
FORDIFFERENTTYPES OFCMOS TECHNOLOGY
leakage current in the simulation, a constant leakage current of 0.8 fA is applied to the capacitor of 2 pF, the stored voltage will gradually decreased. The capacitance of 2 pF is chosen as a compromise between the weight storage time and the capacitor chip area.
The total of 98 exemplar patterns to be processed in the SRMCNN are classified into four groups. Group 1 includes 35 (no. 1 no. 35) Chinese characters with vertical-horizontal lines of two-pixel width. group 2 includes 52 (no. 36 no. 87) English characters (capital and small letters) with the slant lines. Group 3 includes six (no. 88 no. 93) patterns with ver-tical-horizontal grid lines. Group 4 includes five (no. 94 no. 98) patterns with slant lines only. It has the most complicated patterns. Variations of the selected weights in the template during various operation periods in some selected cells are examined to verify the RM phenomenon in the SRMCNN. In Table II(a), the generated ratio weights and of the cells (3,1), (6,2), (9,3), (11,17), and (15,5) after the learning period and the elapsed period with the learned 36 (no. 1 No 36) exemplar patterns are listed. In Table II(b), the weights of the cells (5,10), (8,6), (11,2), (13,16), and (16,12) with learned 98 exemplar patterns. As shown in both Tables II(a) and II(b), the learned templates for different input exemplar patterns are different. The template for larger number learned patterns
LAI AND WU: ARCHITECTURAL DESIGN AND ANALYSIS 1187
TABLE II
GENERATEDRATIOWEIGHTS OFSOMENEURONCELLS IN THE182 18 SRMCNNFOR(a) 36 LEARNEDPATTERNS,AND(b) 98 LEARNEDPATTERNS
AFTERDIFFERENTOPERATIONPERIODS
has fewer elements than that for small number of learned pat-terns. Moreover, the constant leakage current can enhance the larger ratioed weights while suppressing the smaller is to zero. For N larger ratio weights, they are enhanced to 1/N during the elapsed period. The effect is called the feature enhancement effect [12], [13].
Due to the feature enhancement effect, the variations of the ratio weights of two templates of and versus the elapsed time factor is shown in Fig. 5(a) and (b), respectively, during the elapsed period.. The elapsed time factor is normal-ized by the elapsed time of 50 sec. As seen from Fig. 5(a) and (b), the value of the weight is increased to 1 or 1/N whereas the others are decayed to zero. For example, the template weights of the cell , at
is changed to
at as shown in Fig. 5(a). Sim-ilarly, The weights at is changed to at as shown in Fig. 5(b). It is found that the success rate of pattern recognition is related to the elapsed time factor. The minimum required elapsed time factors that
Fig. 5. Variations of the ratioed weights: (a) A and (b) A under constant leakage current.
TABLE III
MATLABSIMULATIONRESULTS OFMINIMUMREQUIREDELAPSEDTIME
FACTOR:FORMAXIMUMSUCCESSRATE OFRECOGNITION
yield the maximum success recognition rate of with different numbers of the learned patterns are given in Table III. The minimum required elapsed time factors are from 8 to 18 for different patterns, which corresponds a range from 400 to 900 s. Note that the maximum elapsed time is generally proportional to the number of the learned patterns and their complexity.
One hundred noisy test patterns and two types of noise are used in simulations to determine the success rate of pattern recognition. One type is the uniform distribution random noise at the levels between 0 and 0.05 , where n is a noise level factor. The other type is the normal distribution random noise
Fig. 6. (a) Input test patterns with uniform noise level of 0.8. (b) Recognized output patterns.
with a noise standard variance of 0.05 , where is a noise variance factor. As verified by the simulation results, the 18 18 SRMCNN can learn 93 patterns and successfully recognize the corresponding 93 noisy patterns of groups 1–3 with uniform (normal) distribution noise at a level of (variance of ). The success rate is 100%. The simulation shows that the learned template already catch the features of all three groups of patterns. Thus, actually more than 93 patterns in the same groups can be recognized correctly. Fig. 6(a) shows some noisy test patterns with uniform noise level of 0.8, whereas Fig. 6(b) shows the correctly recognized patterns. The success rate versus the noise level factor n and the noise variance factor for 93 (no. 1 no. 93) noisy test patterns with uniform and normal distribution noise are shown in Fig. 7(a) and (b), respectively. The figures show that the success rate decreases as the noise level increases beyond 0.8 and the noise variance exceeds 0.3.
The success rate versus the noise level factor and the noise variance factor for 98 noisy patterns of the four groups with
uni-Fig. 7. Success rate versus (a) uniform distribution noise level of 0.05n and (b) normal distribution noise variation of 0.05m for 93 (no. 1 no. 93) noisy test patterns.
form and normal distribution noise shown in the Fig. 8(a) and (b), respectively.Thesuccessrateis97%foruniformnoiselevelsof0.8 and a normal noise variance of 0.25. The rate is rapidly decreased at noise levels over 1.0 or noise variances over 0.25. Analysis indi-cates that the two patterns include only slanted lines within group 4, as shown in Fig. 9, and cannot be completely recognized. Ac-cordingly, the success rate is degraded to 97%. If only five patterns ingroup4arelearnedandrecognizedunderuniformnoise,thesuc-cess rate can reach 100% when the uniform noise level is 0.8, as shown in Fig. 8(c)
Allthesimulationresultsconcerningthesuccessrateforvarious numbers of patterns and different types of noise are summarized in Table IV. Those simulation results indicate that the SRMCNN has a better learning and recognition capability if the learned patterns are simpler and the noise is lower. For complex patterns like those of group 4, the numbers of pattern learning and recognition should be decreased to yield a 100% success rate.
If one pattern with vertical-horizontal lines in group 3 is added to group 4, the success rate is decreased to 90% due to the learning of a different type of pattern from those five patterns with slant lines only.
The patterns not already learned, are included in the noisy patterns to be recognized and to verify the effect of learning on that of recognition in the SRMCNN. It is found that almost no
LAI AND WU: ARCHITECTURAL DESIGN AND ANALYSIS 1189
Fig. 8. Success rate versus (a) uniform distribution noise level of 0.05n for 98 (no. 1 no. 98) noisy test patterns; (b) normal distribution noise variance of 0.05 m for 98 (no. 1 no. 98) noisy test patterns; and (c) uniform distribution noise level of 0.05 n for five (no. 94 no. 98) noisy test patterns.
unlearned patterns can be recognized correctly. Thus, pattern learning is required to recognize a correct pattern.
To investigate the recognition convergence of SRMCNN, a noisy pattern with a uniform noise level of 0.8, as shown in Fig. 10(a) is recognized as the stable pattern in Fig. 10(b). The value corresponding to the energy function of each iteration in the recognition operation during the recognition period is shown in Fig. 10(c). It can be seen that the value of energy function is decreased to the minimum value and the correct pattern is generated after two iterations.
It has been shown that the number of connection weights in the SRMCNN is much less then that in the Hopfield neural net-work, and the SRMCNN can achieve higher capabilities with 93 patterns. The 18 18 SRMCNN has 1620 weight connections while the 18 18 Hopfield network has 104 652. The circuit complexity of SRMCNN is approximately 1/65 of that of the Hopfield network.
For comparisons, conventional CNN associative memories have been proposed with the learned weights of the A template processed without RM and leakage during the recognition oper-ation [5]–[9]. It is shown that the maximum numbers of stored and recognized patterns is 25 (12) for a 9 9 CNN with 49 (25) weight connections. The 18 18 RMCNN without a self-feed-back weight in the A template can learn and recognize five
pat-terns [13]. The proposed SRMCNN with RM and self-feedback weight can enhance the feature of the exemplar patterns and sig-nificantly improve the capability of recognition. As shown in the simulation results, the 18 18 SRMCNN can learn and recog-nize 93 noisy patterns with five weights connection. This veri-fies the improved recognition capability of the SRMCNN.
Using the same learning algorithm but without RM and leakage current, 15 exemplar patterns can be learned in the 18 18 CNN and only 6 (11) patterns could be correctly recognized from input noisy patterns with a uniform noise level of 0.5 (0.3). The success rate of recognition is 40% (73%). This verifies the importance of the effect of RM on the learning and recognition capability of the SRMCNN.
V. CONCLUSION
In this paper, the ratio memory cellular nonlinear network with self-feedback (SRMCNN) is proposed and analyzed. In the SRMCNN, the modified Hebbian learning algorithm with self-feedback is applied to the generation the absolute weights from the sets of input exemplar patterns, and then transform them into ratio weights through the ratio memory to form the coefficients of space-variant A template. With RM and the modified Heb-bian learning algorithm with self-feedback, the SRMCNN can be used as the associative memory for learning, recognizing, and
Fig. 9. Two specific patterns in group 4 with only slant lines. TABLE IV
SUCCESSRATE FORVARIOUSSETS OFLEARNEDPATTERNSWITHNOISE
Including 2 patterns in group 4 as shown in Fig. 10
recovering patterns. The simulation results have shown that the 18 18 SRMCNN with five weights connection can learn and
Fig. 10. (a) Input noisy test pattern. (b) Output stable pattern. (c) Energy function during the recognition period.
recognize 93 noisy patterns with a 100% success rate at a uni-form distribution level of 0.8 and a normal distribution variance of 0.3. This has successfully verified the correct function and superior performance of SRMCNN in the patterns recognition. The proposed SRMCNN with the feature enhancement effect of the RM under constant leakage on the template coefficients can learn and recognize patterns with fewer weight connections than that of the Hopfield neural network. Moreover, the pro-posed SRMCNN with the self-feedback ratio weight can learn and recognize more patterns than the CNN associative mem-ories with RM and without RM, given the same learning al-gorithm and the same constant leakage in the coefficients of space-variant templates. Simulation results have successfully verified the correct function of 18 18 SRMCNN. Since the proposed SRMCNN has the advantages in learning, storing, and recognizing image patterns, it is suitable for appropriate applications of nanoelectronic associative memory systems for real-time image processing.
ACKNOWLEDGMENT
The authors would like to thank the National Science Council of Taiwan, R.O.C., for their support.
REFERENCES
[1] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE
Trans. Circuits Syst., vol. 35, pp. 1257–1272, Oct. 1988.
[2] , “Cellular neural networks: Applications,” IEEE Trans. Circuits
Syst., vol. 35, pp. 1273–1290, Oct. 1988.
[3] T. Roska and J. Vandewalle, Cellular Neural Network. New York: Wiley, 1993, pp. 1–43.
LAI AND WU: ARCHITECTURAL DESIGN AND ANALYSIS 1191
[4] L. O. Chua, CNN: A Paradigm for Complexity, Singapore: World Scien-tific, 1998, ch. 1–2.
[5] D. Liu and A. N. Michel, “Cellular neural networks for associative mem-ories,” IEEE Trans.Circuits Syst. II, vol. 40, pp. 119–121, Apr. 1993. [6] A. Lukianiuk, “Capacity of cellular neural networks as associative
mem-ories,” in Proc. 1996 4th IEEE Int. Workshop Cellular Neural Networks
Applications, 1996, pp. 37–40.
[7] M. Brucoli, L. Carnimeo, and G. Grassi, “An approach to the design of space-varying cellular neural networks for associative memories,” in
Symp. 37th Midwest Circuits Systems, vol. 1, 1994, pp. 549–552.
[8] H. Kawabata, M. Nanba, and Z. Zhang, “On the associative memories in cellular neural networks,” in IEEE Int. Conf. Systems, Man, Cybernetics,
Computational Cybernetics Simulation, vol. 1, 1997, pp. 929–933.
[9] P. Szolgay, I. Szatmari, and K. Laszlo, “A fast fixed point learning method to implement associative memory on CNNs,” IEEE Trans.
Circuits Syst., vol. 44, pp. 362–366, Apr. 1997.
[10] G. Grassi and E. Di Sciascio, “Learning algorithm for pattern classifica-tion using cellular neural network,” Electron. Lett., vol. 36, no. 23, pp. 1941–1943, Nov. 2000.
[11] R. Perfetti and G. Costantini, “Multiplierless digital learning algorithm for cellular neural networks,” IEEE Trans. Circuits Syst., vol. 48, pp. 630–635, May 2001.
[12] C.-Y. Wu and C.-H. Cheng, “The design of cellular neural network with ratio memory for pattern learning and recognition,” in Proc. 6th IEEE
Int. Workshop Cellular Neural Network Applications, May 2000, pp.
301–307.
[13] , “A learnable cellular neural network(CNN) structure with ratio memory for image processing,” IEEE Trans. Circuits Syst. I, vol. 49, pp. 1713–1723, Dec. 2002.
[14] C.-Y. Wu and J.-L. Lai, “The improvement of pattern learning and recog-nition capability in ratio-memory cellular neural networks with nondis-crete-type Hebbian learning algorithm,” in Proc. IEEE Int. Symp.
Cir-cuits Syst., vol. I, May 2002, pp. 629–632.
[15] C.-H. Cheng and C.-Y. Wu, “The design of Ratio-Memory Cellular Neural Network (RMCNN) with self-feedback template weight for pattern learning and recognition,” in Proc. 7th IEEE Int. Workshop
Cellular Neural Network Applications , July 2002, pp. 609–615.
[16] B.-J. Sheu and J. Choi, Neural Information Processing and VLSI. Nor-well, MA: Kluwer, 1995, ch. 2, 5.
Jui-Lin Lai (M’01) received the B.S degree in electronic engineering from the National Taiwan University of Science and Technology, Taipei, Taiwan, R.O.C., and the M.S degree from the Insti-tute of Control Engineering, National Chiao-Tung University, Hsinchu, Taiwan, in 1984 and 1990, respectively, where he is currently working toward the Ph.D. degree at the Institute of Electronic engi-neering.
In 1984, he joined the faculty of the Electronics Engineering Department, National Lien-Ho Institute of Technology, Miao-Li, Taiwan, where, in 2003, the Institute was upgraded and renamed the National United University. His research interests include analog VLSI design, neural networks, and computer architecture.
Peter Chung-Yu Wu (S’76–M’76–SM’96–F’98) was born in 1950. He received the M.S. and Ph.D. degrees from the Department of Electronics Engi-neering, National Chiao-Tung University (NCTU), Hsinchu, Taiwan, R.O.C., in 1976 and 1980, respec-tively.
Since 1980, he has served as a Consultant to high-tech industry and research organizations and has built strong research collaborations with high-tech indus-tries. From 1980 to 1983, he was an Associate Pro-fessor at the NCTU. From 1984 to 1986, he was a Visiting Associate Professor in the Department of Electrical Engineering, Port-land State University, PortPort-land, OR. Since 1987, he has been a Professor at the NCTU, and from 1991 to 1995, he served as the Director of the Division of Engineering and Applied Science on the National Science Council, Taiwan. In addition, he conducted visiting research at University of California, Berkeley in Summer 2002. He was honored as the Centennial Honorary Chair Professor at National Chiao Tung University, from 1996 to 1998. Currently, he is the Chair Professor at National Chiao Tung University and Adjunct International Pro-fessor at University of Illinois at Urbana-Champaign. He has published more than 250 technical papers in international journals and conferences. He also has 19 patents including nine U.S. patents. His research interests include nano-electronics and VLSI including circuits and systems in low-power/low-voltage mixed-signal design and systems, biochips, neural vision sensors, RF circuits, and CAD analysis.
Dr. Wu was a recipient of the IEEE Fellow Award in 1998 and the Third Millennium Medal in 2000. In Taiwan, he has received numerous research awards from the Ministry of Education, National Science Council, and profes-sional foundations. He is a member of Eta Kappa Nu and Phi Tau Phi Honorary Scholastic Societies.
