A Recurrent Fuzzy Coupled Cellular Neural Network
System With Automatic Structure and
Template Learning
Chun-Lung Chang, Kan-Wei Fan, I-Fang Chung, and Chin-Teng Lin, Fellow, IEEE
Abstract—The cellular neural network (CNN) is a powerful
technique to mimic the local function of biological neural cir-cuits, especially the human visual pathway system, for real-time image and video processing. Recently, many studies show that an integrated CNN system can solve more complex high-level intelligent problems. In this brief, we extend our previously pro-posed multi-CNN integrated system, called recurrent fuzzy CNN (RFCNN) which considers uncoupled CNNs only, to automatically learn the proper network structure and parameters simultane-ously of coupled CNNs, which is called recurrent fuzzy coupled CNN (RFCCNN). The proposed RFCCNN provides a solution to the current dilemma on the decision of templates and/or fuzzy rules in the existing integrated (fuzzy) CNN systems. For compar-ison, the capability of the proposed RFCCNN is demonstrated on the same defect inspection problems. Simulation results show that the proposed RFCCNN outperforms the RFCNN.
Index Terms—Cellular neural network (CNN) template design,
defect inspection, fuzzy clustering, fuzzy neural network.
I. INTRODUCTION
T
HE two-dimensional (2-D) inputs and outputs of the cel-lular neural network (CNN) [1], [2] make it very suitable for image processing. Besides some basic image processing tasks, the CNN has been used to mimic the local function of biological neural circuits, especially the human visual pathway system [3]. A current biological study [4] shows that mammalian visual systems process the world through a set of separate parallel channels. Each subchannel can be regarded as a unique CNN. The output of these subchannels is then combined to form the new channel responses. As a result, it is widely accepted that using a set of CNNs in parallel can achieve higher level information processing and reasoning functions either from biologics or application points of views. Such an integrated CNN system can solve more complex intelligent problems.For designing an integrated CNN system, in addition to the determination of a set of templates, another kernel problem is
Manuscript received July 5, 2005; revised November 10, 2005. This paper was recommended by Associate Editor H. Leung.
C.-L. Chang is with the Mechanical and System Research Laboratories, In-dustrial Technology Research Institute, Hsinchu, Taiwan 310, R.O.C.
C.-T. Lin and K.-W. Fan are with the Department of Electrical and Control Engineering, National Chiao-Tung University, Hsinchu 300, Taiwan, R.O.C. (e-mail: [email protected]).
I.-F. Chung is with the Institute of Bioinformatics, National Yang-Ming Uni-versity, Taipei 112, Taiwan, R.O.C. (e-mail: [email protected]).
Digital Object Identifier 10.1109/TCSII.2006.876388
the way of integration. To solve this problem, the fuzzy in-ference system (FIS) can play an important role to integrate a set of CNNs into a system. To make a set of CNNs in parallel achieve higher level information processing, several integrated CNN systems are proposed [4]–[7]. They have two common characteristics. First, they all used many CNNs in parallel to solve a complex problem. Second, they all used FIS to make a decision. The common drawbacks of these approaches are that they all need to assign the corresponding templates of CNNs in advance (i.e., templates cannot be learned) and they all need to take the fuzzy rules manually by domain experts. Although, according to Nossek’s survey [8], the template coefficients of a CNN can be found by design [8], [9] or by learning [8], [10], these techniques cannot be applied to the design or learning of an integrated CNN system directly.
To cope with these drawbacks, we proposed a novel framework for automatically constructing a multiple-CNN integrated neural system in the form of a recurrent fuzzy neural network (FNN) [11], [12]. This system, called recurrent fuzzy CNN (RFCNN) [13], can automatically learn its proper network structure and parameters simultaneously. The structure learning includes the fuzzy division of the problem domain and the creation of fuzzy rules and CNNs. The parameter learning includes the tuning of fuzzy membership functions and CNN templates. In the RFCNN, each learned fuzzy rule corresponds to a CNN. Hence, each CNN takes care of a fuzzily separated problem region, and the functions of all CNNs are integrated through the fuzzy inference mechanism. However, since the RFCNN only considers the learning of uncoupled CNN templates, its ability to solve more complex high-level machine vision problems is quite limited. In this brief, we extend our previously proposed RFCNN to automatically learn the proper network structure and parameters simultaneously of coupled CNNs. Due to the inclusion of coupled CNNs, the proposed recurrent fuzzy coupled CNN (RFCCNN) has shown more powerful abilities in detecting the fine detailed defects of color filters, compared with the previous RFCNN, in the simulations.
This brief is organized as follows. Section II describes the structure and functions of the proposed RFCCNN. Section III briefly describes the online learning algorithm for the RFCCNN. Section IV gives simulation results and discussions. Finally, conclusions are summarized in Section V.
II. STRUCTURE OF THERFCCNN
Here, the structure of the proposed RFCCNN shown in Fig. 1 is introduced. For clarity, we consider a CNN, with time constant , time step , and neighborhood within a
Fig. 1. Structure of the proposed RFCCNN.
radius , which is characterized by the following templates:
(1) where , , and are the feedback template, control tem-plate, and bias of the th CNN, respectively. By defining a CNN as above, the six-layered RFCCNN network will realize a fuzzy model of the following form:
Rule IF is and is and is
THEN is (2)
or
Rule IF is and is and is THEN is
(3) where the current input vector is ,
is ,
is , is a
bipolar sigmoid function, is a fuzzy set, and , , and are consequent parameters representing feedback template, control template, and bias of the th CNN, respectively. The RFCCNN is a six-layered network structure with one feedback layer and can automatically learn its proper network structure (the creation of fuzzy rules and CNNs) and parameters (the tuning of fuzzy membership functions and CNN templates) simultaneously. In this brief, as defined in (1)–(3), we extend the RFCNN to RFCCNN including the structure and parameter learning of coupled CNN cells. The six-layered network struc-ture of the RFCCNN is mostly the same as that of the RFCNN,
except for the feedback layer. In the feedback layer of the RFCCNN, the feedbacks are from coupled CNNs, i.e., is , but, in the feedback layer of the original RFCNN, the feedbacks are from uncoupled CNNs only, i.e., is . The details of the function of each node of the RFCNN are described in [13].
III. LEARNINGALGORITHMS FOR THERFCCNN Two types of learning, structure and parameter learning, are used concurrently for the RFCCNN. In the RFCCNN, the struc-ture learning includes the fuzzy division of the problem domain (precondition structure identification) and the creation of fuzzy rules and CNNs (consequent structure identification). The pre-condition structure identification corresponds to the input-space partitioning. As to the consequent structure identification, the main task is to decide when to generate a new consequent term (or a new CNN) for the output variable. In our system, we pro-posed an online independent component analysis (ICA) mixture model [13] to realize the precondition and consequent structure identification part of the RFCCNN.
For the parameter learning, the parameters of each CNN tem-plate in the consequent parts are adjusted by the ordered deriva-tive algorithm to minimize a given cost function. The parameters in the precondition part are adjusted by the online ICA mixture model algorithm. The RFCCNN can be used for normal oper-ation at any time during the learning process without repeated training on the input–output patterns when online operation is required. There are no rules (i.e., no nodes in the network except the input–output nodes) in this network initially. They are cre-ated dynamically as learning proceeds upon receiving online in-coming training data by performing the following learning pro-cesses simultaneously. The details of these learning propro-cesses are described in the remainder of this section.
A. Structure Learning Algorithm of RFCCNN
Efficient partition of input–output data will result in faster convergence and better performance for the RFCCNN. The most direct way is to partition the input space into grid types, with each grid representing a fuzzy if–then rule. This is called grid-based partitioning. The major problem of this kind of partitioning is that the number of fuzzy rules (and thus the number of CNNs) increases exponentially if the number of input variables or that of the partitions increases. A flexible partition method, such as the clustering-based approach which clusters the input training vectors in the input space, will reduce the rule and CNN numbers. In this brief, our proposed clustering method based on a new online ICA mixture model is used to provide a better partition of the input–output space for the proposed RFCCNN. The background and algorithm of the proposed online ICA mixture model for clustering can be found in [13] and [14].
B. Parameter Learning Algorithm of RFCCNN
After the network structure is adjusted according to the cur-rent training pattern, the network then enters the parameter iden-tification phase to adjust the parameters of the network opti-mally based on the same training pattern. Notice that the fol-lowing parameter learning is performed on the whole network after structure learning, regardless of whether the nodes (links)
are newly added or already existed. Since the RFCCNN is a dynamic system with feedback connections, the back propaga-tion learning algorithm cannot be applied to it directly. Also, due to the online learning property of the RFCCNN, the offline learning algorithms for the recurrent neural networks, like back propagation through time and time-dependent recurrent back propagation [12], cannot be applied here. Instead, the ordered derivative [15], which is a partial derivative whose constant and varying terms are defined using an ordered set of equations, is used to derive our learning algorithm. The ordered set of equa-tions, described in Section II in each layer, is summarized in (5)–(10). Our goal is to minimize the error function
(4) where is the desired output, is the cur-rent output, and is . For each training data set, starting at the input nodes, a forward pass is used to compute the activity levels of all the nodes in the net-work to obtain the current output . In the following, dependency on time will be omitted unless emphasis on tem-poral relationships is required.
Summarizing the node functions defined in Section II, the function performed by the network is
(5) (6) where (7) (8) (9) and (1) is redefined as the following equation for clarity:
(10) With the above formula and the error function defined in (4), we can derive the update rules for the free parameters in the RFCCNN as follows.
The update rule of (the parameter of feedback template of the th CNN) is (11) (12) where (13) (14) where (15) (16) Hence, the parameter is updated by
(17) Similarly, the parameter (the parameters of control tem-plate of the th CNN) is updated by
(18) and the parameter (the bias of the th CNN) is updated by
(19) As shown in (14)–(16), the update rules are in recursive form. The value ( ) is equal to zero initially. For the remaining free parameters in the RFCCNN, they are obtained during the structure-learning phase by the online ICA mixture model algo-rithm proposed in [13]. Notice that, according to the real-time recurrent learning (RTRL) scheme, we can also obtain the same parameter learning rules for the RFCCNN.
IV. SIMULATIONRESULTS ANDDISCUSSIONS
The capability of the proposed RFCCNN is demonstrated on the real-world defect inspection problems. Here we are inter-ested in the defect inspection of color filter, which is one of components in TFT-LCD array process and gives each pixel of LCD its own color. The difficulties in the defect inspection of color filter are its complex texture and demand for high-speed processing. For the reasons of high-speed processing, and that different kinds of defects in the color filter need different CNN templates and some complex defects cannot be detected by a single CNN, the proposed RFCCNN is a good alternative to de-tect defects in color filter images. To train the RFCCNN, we use a 3 3 window to get the system inputs and set the whole image as the inputs of the RFCCNN. The 3 3 window covers the cen-tral pixel and its eight connected neighbors. The training image and corresponding desired output are shown in Fig. 2(a) and (b). As mentioned in Section III, there are no rules (and no CNNs) in the RFCCNN initially. They are created dynamically as learning proceeds upon receiving online incoming training data by per-forming the learning processes. When the learning processes are done, six clusters (six fuzzy rules and CNN templates) were ob-tained. For the example of color filter, it takes approximately 90 s to learn the structure (interconnection set) and the parameters with a Pentium IV 2.0-G PC. However, the training can be done offline, so it is not a problem for the online processing of CNN. For simulation in this brief, it causes approximately 9 s. After the proposed RFCCNN is implemented by analog circuits in the
Fig. 2. Training images. (a) Input image. (b) Desired output.
Fig. 3. The outputs of layers 3 and 4 and the feedback layer for the training image. (a) The output of the RFCCNN. (b)–(g) The outputs of the six layer-4 nodes, respectively. (h)–(m) The outputs of the six CNNs in the feedback layer, respectively. (n)–(s) The outputs of the six layer-3 nodes, respectively (firing strength of each rule).
future, we believe that the processing times will be much faster than those of the simulation.
Fig. 3 shows the output of RFCCNN and the outputs of layers 3 and 4 and the feedback layer for the training image. Fig. 3(a) shows the output of the RFCCNN. Fig. 3(b)–(g) shows the out-puts of the six layer-4 nodes, respectively, i.e., the outout-puts of the six CNNs in the feedback layer multiplied by the outputs of the six layer-3 nodes (i.e., firing strength of each rule), respectively. Fig. 3(h)–(m) shows the outputs of the six CNNs in the feed-back layer, respectively. Fig. 3(n)–(s) shows the outputs of the six layer-3 nodes, respectively (firing strength of each rule). The sum of the outputs of the six layer-4 nodes [i.e., Fig. 3(b)–(g)] forms the RFCCNN final output [Fig. 3(a)]. From Fig. 3(b)–(g), we can see that CNNs 4 and 5 take care of the defect texture in the right side of the training image, CNNs 1 and 6 mainly take care of the defect textures in the left side of the training image, and the other CNNs balance the output of the RFCCNN. The template of each learned CNN is given as
Fig. 4. Simulation (testing) results of the learned RFCCNN. (a), (d), (g) Input testing images. (b), (e), (h) Corresponding detection results of RFCCNN. (c), (f), (i) Corresponding detection results of an uncoupled RFCNN.
(20) Based on the learned structure and parameters of the RFCCNN, we test several images and three of those images as shown in Fig. 4. Fig. 4(a), (d), and (g) shows input testing images. Fig. 4(b), (e), and (h) displays the corresponding detection results of the RFCCNN. Fig. 4(c), (f), and (i) shows the corresponding detection results of the uncoupled RFCNN. From Fig. 4(b), (e), and (h), we can see that the learned structure and CNN templates of the RFCCNN are well suited to detect the defects of color filer images. It has also been confirmed that detection results are still good if the images are shifted or rotated. One of simulation results is shown in Fig. 5.
From Fig. 4, we can see some differences between the RFCCNN (coupled RFCNN) and the uncoupled RFCNN for
Fig. 5. Simulation results of shifted and rotated images. (a), (c), (e) Input testing images of original, shifted, and rotated ones, respectively. (b), (d), (e) Corresponding detection results of the RFCCNN.
TABLE I
COMPARISON OFDETECTIONRATE
defect inspection of a color filter. First, as shown in Fig. 4(g)–(i), the results of the coupled RFCNN is better than those of uncou-pled RFCNN for detecting the large defects on the top-left of test image shown in Fig. 4(g). Second, as shown in all test images and corresponding detection results, the results of the coupled RFCNN are better than those for the uncoupled RFCNN for de-tecting the black defects. The results of the coupled RFCNN are better for detecting the large defects in that the coupled RFCNN, like the coupled CNN, has fully output feedback and will take care of further neighboring pixels. Some quantity comparisons are shown in Table I. Here detection rate is defined as the ratio of detected defect pixel number to real defect pixel number. As we can see from Table I, the detection rates of the RFCCNN are better than those of the RFCNN, regardless of all defects, large defects, or black defects.
The main idea of the proposed RFCCNN is an integrated system of FIS and CNNs, which can construct fuzzy rules and CNN templates automatically. A scheme to implement the RFCCNN is described as follows. The circuit design technique of a multilayer CNN (MLCNN) [16] can be applied to imple-ment the multilayer structure of the proposed RFCCNN model in Fig. 1. A CNN-based Gaussian function circuit as designed in [17] can realize the Gaussian membership function required in layer 2. The fuzzy logic operations in layers 3 and 4 can be realized by analog CNN circuits as studied in [7] and [18]. Two other layers correspond to the CNN’s input and output, respectively. Therefore, it is very promising and feasible to implement the RFCCNN using high-speed analog circuits.
V. CONCLUSION
In this brief, we extended our previous work, called the RFCNN, from considering uncoupled CNNs to coupled CNNs, called the RFCCNN, for automatically constructing a multi-CNN integrated neural system. This CNN-based fuzzy neural network can automatically learn its proper network structure and parameters simultaneously. In order to verify the capability of the RFCCNN, a defect inspection problem has been demonstrated and compared with the RFCNN. The simulation results show that the performance of the proposed RFCCNN is better than that of the RFCNN. In addition, a scheme for hardware realization of the RFCCNN has been proposed. We believe that such an integrated CNN system, the RFCCNN, has potential to solve more complex intelligent problems, and our future work is to apply it to more complex real-world problems.
REFERENCES
[1] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE
Trans. Circuits Syst., vol. 35, no. 10, pp. 1257–1272, Oct. 1988.
[2] ——, “Cellular neural networks: Applications,” IEEE Trans. Circuits
Syst., vol. 35, no. 10, pp. 1273–1290, Oct. 1988.
[3] J. Hámori and T. Roska, Receptive field atlas of the retinotopic visual pathway and some other sensory organs using dynamic cellular net-work models, Analogical and Neural Comput. Lab., Budapest, Hun-gary, MTA-SZTAKI, DNS-8-2000.
[4] L. Szatmári, D. Bálya, G. Tímár, C. Rekeczky, and T. Roska, “Multi-channel spatio-temporal topographic processing for visual search and navigation,” in Proc. SPIE Microtechnologies New Millenium, Gran Canaria, Spain, May 2003, vol. 5119-38 , pp. 297–306.
[5] F. Colodro and A. Torralba, “Cellular neuro-fuzzy networks (CNFNs) a new class of cellular networks,” in Proc. 5th IEEE Int. Conf. Fuzzy
Syst., 1996, vol. 1, pp. 8–11.
[6] C. Rekeczky, T. Roska, and A. Ushida, “CNN-based difference-con-trolled adaptive nonlinear image filters,” Int. J. Circuit Theory Appl., vol. 26, pp. 375–423, Jul.-Aug. 1998.
[7] C. Rekeczky, Á. Tahy, Z. Végh, and T. Roska, “CNN based spatio-tem-poral nonlinear filtering and endocardial boundary detection in echocardiography,” Int. J. Circuit Theory Appl., vol. 27, pp. 171–207, Jan.-Feb. 1999.
[8] J. A. Nossek, “Design and learning with cellular neural networks,” Int.
J. Circuit Theory Appl., no. 24, pp. 15–24, 1996.
[9] A. Zarandy, “The art of CNN template design,” Int. J. Circuit Theory
Appl., no. 27, pp. 5–23, 1999.
[10] T. Kozek, T. Roska, and L. O. Chua, “Genetic algorithm for CNN tem-plate learning,” IEEE Trans. Circuits Syst. I: Fundam. Theory Appl., vol. 40, no. 6, pp. 392–402, 1993.
[11] C. T. Lin, Neural Fuzzy Control Systems with Structure and Parameter
Learning. New York: World Scientific, 1994.
[12] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neural-Fuzzy
Syn-ergism to Intelligent Systems. Englewood Cliffs, NJ: Prentice-Hall, 1996.
[13] C. T. Lin, C. L. Chang, and W. C. Cheng, “A recurrent fuzzy cellular neural network system with automatic structure and template learning,”
IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 5, pp. 1024–1035,
May 2004.
[14] C. T. Lin, W. C. Cheng, and S. F. Liang, “An on-line ICA-mixture-model-based self-constructing neural fuzzy network,” IEEE Trans.
Cir-cuits Syst. Part I: Reg. Papers, vol. 52, no. 1, pp. 207–221, Jan. 2005.
[15] P. Werbos, “Beyond regression: New tools for prediction and analysis in the behavior sciences,” Ph.D. dissertation, DEPT. OF ???, Harvard Univ., Cambridge, MA, 1974.
[16] G. Liu and S. Oe, “Texture image segmentation method based on mul-tilayer CNN,” in Proc. 12th IEEE Int. Conf. Tools with Artif. Intell.
(ICTAI’00), 2000, pp. 147–150.
[17] S. A. Chen, J. F. Chung, S. F. Liang, and C. T. Lin, “Hybrid-order texture boundary detection based on CNN circuit models,” in Proc.
IEEE Int. Workshop BioMedical Circuits Syst. (BioCAS’04),
Singa-pore, 2004.
[18] M. Balsi and F. Voci, “Implementation of fuzzy rule based image pro-cessing on the CNN universal machine,” in Proc. Eur. Conf. Circuits
