Cellular Neural Networks for Gray Image Noise Cancellation Based on a Hybrid Linear Matrix Inequality and Particle Swarm Optimization Approach

DOI 10.1007/s11063-010-9150-0

Cellular Neural Networks for Gray Image Noise

Cancellation Based on a Hybrid Linear Matrix Inequality

and Particle Swarm Optimization Approach

Te-Jen Su· Ming-Yuan Huang · Chia-Ling Hou · Yu-Jen Lin

Published online: 2 September 2010

© Springer Science+Business Media, LLC. 2010

Abstract This paper describes a technique for gray image noise cancellation. This method employs linear matrix inequality (LMI) and particle swarm optimization (PSO) based on cellular neural networks (CNN).We use two images that one is desired image and the other is corrupted to find the CNN template. The Lyapunov stability theorem is employed to derive the criterion for uniqueness and global asymptotic stability of the CNN equilibrium point. The current study characterizes the template design problem as a standard LMI problem and the optimization parameters of the templates are carried out by PSO. Finally, the examples are given to illustrate the effectiveness of the proposed method.

Keywords Cellular neural networks· Particle swarm optimization · Linear matrix inequality· Noise cancellation · Image

This study was supported financially in part by grants from the NSC-2009-2221-E-151-057, ROC. T.-J. Su (

B

Department of Electronic Engineering, National Kaohsiung University of Applied Sciences, Kaohsiung 807, Taiwan, ROC

e-mail: [email protected] M.-Y. Huang e-mail: [email protected] C.-L. Hou e-mail: [email protected] Y.-J. Lin e-mail: [email protected] T.-J. Su

1 Introduction

L.O. Chua and L. Yang first introduced cellular neural networks (CNN) in 1988 [2,3]. Due to their local connectivity, using a set of CNNs in parallel to achieve higher-level information processing and reasoning functions is widely accepted in both practical applications and in biologics. This integrated CNN system helps to solve more complex intelligence problems. The key to investigating CNN is finding accurate templates. CNN template design prob-lems for image processing have recently received considerable attention. Researches have taken advantage of both genetic algorithm and multilayer CNN methods to obtain templates for image processing [12]. A previous study [13] employed CNN with a particular hyster-esis nonlinear cell characteristic for image processing. One drawback of the CNN template design is that the templates must be simplified to decrease operation time [12] or to facilitate mathematical analysis of dynamic behavior [4,13,14].

Several studies have recently dealt with CNN stability by choosing various Lyapunov functions [6,8,10,17,18]. Linear Matrix Inequality (LMI) methods have attracted a great deal of attention the past few years because of their computational tractability and usefulness in controlling engineering applications. The number of control problems that can be formu-lated as LMI problems is large and continues to grow [9]. In addition, LMI problems can now be solved efficiently using the powerful MATLAB LMI Toolbox [1]. Particle Swarm Optimization (PSO) was first presented by James Kennedy and Russell Eberhart in 1995 [5]. This method is inspired by observing the behavior of flocking birds and schooling fish. The method consists of a relatively new family of heuristic algorithms that can be used to find optimal (or near optimal) solutions to numerical and qualitative problems. PSO is easily implemented in most programming languages (the core of the algorithm can be written as a single line of code, albeit a long one) and has proven to be very effective and quick for a diverse set of optimization problems.

This study utilized the CNN system with hybrid LMI and PSO for gray image noise can-cellation. Based on the Lyapunov stability, the CNN templates are trained by a given training sample with a smaller size obtained from LMI and PSO. The given CNN templates are then employed to eliminate noise from arbitrary larger corrupted images.

The similar results had been published at International Conference on New Trends in Information and Service Science [11]. But in this paper, the performance of reconstructed images is more improved by modify programs. And the results of the proposed method are compared with the median filter method and the better performance is obtained.

The rest of this paper is organized as follows. Section1 introduces the work of this study. Sections2and3provide basic definitions and preliminary mathematics of the LMI and PSO for thesis development. Section4describes stability conditions for the templates via the Lyapunov stability theorem, and that of CNNs by a hybrid LMI and PSO method. Section5gives examples to demonstrate the proposed methodology. Finally, Sect.6offers some conclusions and recommendations for future work.

2 Linear Matrix Inequality

The LMI approach has gained much attention over the past few years for its computational tractability and usefulness in controlling engineering, and the number of control problems that can be formulated as LMI problems is large and continues to grow [9]. Any LMI can be expressed in the following canonical form:

F(x) = F0+ x1F1+ . . . + xNFN < 0 (1)

where Fi∈ Rn×n, 0 ≤ i ≤ N are the given symmetric matrices; and x = [x1, x2, . . . , xN]T ∈ RN_{is a variable vector. The inequality symbol in (1) means that F(x) is negative-definite.}

Furthermore, multiple LMIs A1(x) > 0, A2(x) > 0, . . . , AM(x) > 0, can be expressed

as the single LMI di ag(A1(x) > 0, A2(x) > 0, . . . , AM(x)) > 0.

In general, many engineering problems can be formulated as one of several types of LMI problems described below.

2.1 The Feasibility Problem

Find x = [x1, x2, . . . , xN]T ∈ RN that satisfies the LMI system (1). This problem can be

solved using the function “feasp” of the LMI Toolbox. The “feasp” function has been recently employed for the problem of stability testing in dynamical systems. For example, based on the control theory, a system ˙x(t) = Ax(t) is stable if and only if there exists a symmetric and positive-definite matrix P> 0, such that the inequality

ATP+ P A < 0 (2)

is satisfied. The “feasp” function can be used to solve this type of problem. 2.2 The Linear Optimization Problem

This study used the problem of minimizing a linear objective, under LMI constraints (MINCX), to design the equalizers for linear and nonlinear channels. The MINCX prob-lem is stated as follows:

Minimize cTx over x ∈ RN subject to F(x) < 0, (3) where c∈ RN is a given vector. This problem can be solved using the “mincx” function of the LMI Toolbox. For example, consider the following optimization problem:

Minimize Trace(X) subject to ATX+ X A + X B BTX+ Q < 0, (4) where A, B, and Q are given matrices.

This optimization problem (4) is obviously equivalent to the following linear objective minimization problem:  ATX+ X A + Q X B BT_{X −I}  < 0 (5)

and can therefore be solved using the “mincx” function. 2.3 The Generalized Eigenvalue Minimization Problem

This problem can be solved using the “gevp” function of the LMI toolbox. The “gevp” function is employed for the decay rate problem.

Minimizeλ subject to F1(x) < λF2(x) (6) where F1(x) < λF2(x) denotes the system of LMI. For example, given A1, A2. Consider the problem of finding a single Lyapunov function V(x) = xTP x, that proves the stability of

and maximizes the delay rate−dV (x)/dt = −α. This problem is equivalent to minimizing

α subject to

I < P, (8)

AT₁P+ P A1 < αP, (9)

AT₂P+ P A2 < αP. (10)

3 Particle Swarm Optimization

The PSO method is inspired by observing the behavior of flocking birds and schooling fish. An individual bird in a flock of birds can learn from past experience to modify its flying speed and direction. In other words, each particle can benefit from its experience when exploring new regions of the search space. The basic concept behind the PSO technique consists of chang-ing the velocity or acceleration of each particle in relation to certain positions (its pbest and

gbest positions) in each time step. Accordingly, each particle has a memory, which allows it to

remember the best positions it has visited within the feasible search space. This best position is commonly called the pbest. Another best value tracked by the particle swarm optimizer is the best value obtained so far by any particle in the neighborhood of the particle. This best location is commonly called the gbest. The behavior of each individual particle is affected by either the local best or the global best particle, which helps it to fly in the search space explor-ing for better solutions. This means that each particle tries to modify its current position and velocity according to the distance between its current position and the pbest, and the distance between its current position and gbest. In PSO, suppose that the search space is D-dimen-sional, and that the i th particle is represented by Xi= (xi 1, xi 2, . . . , xi D). The velocity (rate

of the position change) of this particle is denoted as Vi = (vi 1, vi 2, . . . , vi D). The best

pre-vious position of the i th particle is represented as Pi = (pi 1, pi 2, . . . , pi D). In other words, Pi involves the best previous position which Xi has visited (the best position is

called pbest). The index of the best particle among all the particles in the swarm is defined as the symbol g (called gbest). The particles are manipulated according to the equations below. In its canonical form, Particle Swarm Optimization is modeled by [7,15,16]

vi d(t + 1) = w∗vi d(t) + c1∗r and()1∗(pi d− xi d) + c2∗r and()2∗(pgd− xi d) (11)

xi d(t + 1) = xi d(t) + vi d(t + 1), (12)

wherevi d(t + 1) vi d(t + 1): velocity of particle i at iteration t+1, vi d(t): velocity of particle i at iteration t, xi d(t + 1): position of particle i at iteration t+1, xi d(t): position of particle i

at iteration t, c1: acceleration coefficient related to pbest, c2: acceleration coefficient related to gbest, r and()1: random number uniform distribution U(0, 1), rand()2: random number uniform distribution U(0, 1), pi d: pbest position of particle i , pgd: gbest position of swarm, w: inertia weight.

The general PSO algorithm can be applied to any optimization problem. Figure1shows a flowchart of the PSO algorithm.

Fig. 1 Flowchart of PSO

4 CNN Training by Hybrid LMI and PSO

A CNN system is designed to find one or more templates that realize a certain input-output behavior. This study utilized the CNN system Hybrid LMI and PSO for gray image noise cancellation. Figure2shows a block diagram of the training system. The CNN templates are trained by a given training sample with a smaller size obtained from LMI and PSO. The given CNN templates are then employed to eliminate noise from arbitrary larger corrupted images.

4.1 A Standard CNN

An M by N standard CNN is defined by an M by N rectangular array of cells C(i, j) located at site (i, j), i= 1, 2, . . ., M, j = 1, 2, . . ., N. Each cell C(i, j) is defined mathematically by

˙xi j = −xi j+  C(k,l)∈Nr(i, j) A(i, j; k, l)ykl+  C(k,l)∈Nr(i, j) B(i, j; k, l)ukl+ Ii j, (13)

where xi j ∈ R, ykl ∈ R, ukl ∈ R, and Ii j ∈ R are the state, output, input, and threshold of

cell C(i, j), respectively; A(i, j; k, l) is the feedback matrix; and B(i, j; k, l) is the input matrix. The output yi jis a memory-less nonlinear function of the state xi j, depicted in Fig.3

yi j = f (xi j) =

1

2xi j+ 1 − xi j− 1. (14) Figure4shows the signal flow structure of a CNN with a 3× 3 neighborhood. The two shaded cones symbolize the weighted contributions of the input and output values of ell C(k, l)∈ N(i, j) to the state value of the center cell C(i, j).

The given input and initial condition the template coefficients completely define the net-work behavior. All cells in the CNN are assumed to have equal parameters and hence equal templates (space invariance). The term, cloning templates is used to emphasize this invari-ance property. This means that the set of(2r + 1) × (2r + 1) are positive integer numbers,

Fig. 3 Output nonlinearity

where r is a positive integer number [4], A(i, j; k, l), B(i, j; k, l) and I, completely determine the behavior of an arbitrary large two-dimensional CNN.

The templates are often expressed in compact form by means of tables or matrices. For instance, the following two square matrices are used for a CNN with r= 1:

A= ⎛

⎝a(i − 1, j − 1) a(i − 1, j) a(i − 1, j + 1)a(i, j − 1) a(i, j) a(i, j + 1) a(i + 1, j − 1) a(i + 1, j) a(i + 1, j + 1)

⎞ ⎠ , B=

⎛

⎝b(i − 1, j − 1) b(i − 1, j) b(i − 1, j + 1)b(i, j − 1) b(i, j) b(i, j + 1) b(i + 1, j − 1) b(i + 1, j) b(i + 1, j + 1) ⎞ ⎠ .  C(k,l)∈Nr(i, j) A(i, j; k, l)ykl =  |k−i|≤1  |l− j|≤1 A(k − i, l − j)ykl = a−1,−1yi−1, j−1+ a−1,0yi−1, j+ a−1,1yi−1, j+1 + a0,−1yi, j−1+ a0,0yi, j+ a0,1yi, j+1 + a1,−1yi_{+1, j−1}+ a1,0yi_{+1, j}+ a1,1yi_{+1, j+1} = 1  k=−1 1  l=−1 ak,lyi+k, j+l = a_−1,−1 a_−1,0 a_−1,1 a0,−1 a0,0 a0,1 a1,−1 a1,0 a1,1 ∗ yi−1, j−1 yi−1, j yi−1, j+1 yi, j−1 yi, j yi, j+1 yi_{+1, j−1} yi_{+1, j} yi_{+1, j+1} = A∗Yi j,

where the 3×3 matrix A is called the feedback cloning template, and the symbol “*” denotes the summation of dot products, henceforth called a template dot product. In discrete mathe-matics, this operation is called “spatial convolution”. The 3× 3 matrix Yij can be obtained by moving an opaque mask with a 3× 3 window to position (i, j) of the M by N output image Y, henceforth called the output image at C(i, j).

 C(k,l)∈Nr(i, j) B(i, j; k, l)ukl =  |k−i|≤1  |l− j|≤1 B(k − i, l − j)ukl = b−1,−1ui−1, j−1+ b−1,0ui−1, j+ b−1,1ui−1, j+1 + b0,−1ui, j−1+ b0,0ui, j+ b0,1ui, j+1 + b1,−1ui_{+1, j−1}+ b1,0ui_{+1, j} + b1,1ui_{+1, j+1} = 1  k=−1 1  l=−1 bk,lui+k, j+l = b_−1,−1 b_−1,0 b_−1,1 b0,−1 b0,0 b0,1 b1,−1 b1,0 b1,1 ∗ ui−1, j−1ui−1, j ui−1, j+1 ui, j−1 ui, j ui, j+1 ui_{+1, j−1}ui_{+1, j} ui_{+1, j+1} = B∗Ui j,

where the 3× 3 matrix B is called the feed forward or input control template, and Ui jis the

translated masked input image. Figure5shows there are typically three ways to order the variables.

Table 4 PSNR of LMI and

PSO-CNN for 20% Noise Fig. 12 Fig. 13

Salt and Pepper 12.1253 dB 11.5312 dB

Median Filter 26.15 dB 26.06 dB

LMI and PSO-CNN 35.89 dB 35.03 dB

P S N R = 10 log₁₀255

2

MSEdB (37)

where˜yi jrepresents pixels in the ideal image;ˆyi jrepresents pixels in the reconstructed image

as the CNN output.

6 Conclusions

This study proposes a solution for CNN template design to remove noise in gray images. This research demonstrates that the design problem can be transformed into LMIs, and the optimization parameters of the templates can be obtained by PSO. This work, LMI-CNN-PSO, is first applied for treating the noise cancellation of gray images. Compared with the median filter method, the results show that the new strategy provides a better way to handle the gray noise cancellation problem.

By the way, the design templates can eliminate noise from arbitrary corrupted images in real system. We believe the proposed method also to be applied to the satellite image reconstruction, and different types of the noises cancellation will be studied in the future.

References

1. Boyd S, Ghaoui LE, Feron E, Balakrishnan V (1994) Linear matrix inequalities in system and control theory, studies in applied mathematics. SIAM, Philadelphia

2. Chua LO, Yang L (1988) Cellular neural networks: theory. IEEE Trans Circuits Syst 35:1257–1272 3. Chua LO, Yang L (1988) Cellular neural networks: applications. IEEE Trans Circuits Syst 35:1273–1290 4. Chua LO, Roska T (2002) Cellular neural networks and visual computing. Cambridge University Press,

New York

5. Gahinet P, Nemirovskii A, Laub A, Chilali M (1995) LMI control toolbox: for use with MATLAB. The MATH Works Inc., Portola Valley

6. Gilli M (1993) A Lyapunov function approach to the study of the stability of cellular neural networks. IEEE Int Symp Circuits Syst 4:2584–2587

7. Heppner F, Grenander U (1990) A stochastic nonlinear model for coordinated bird flocks, the ubiquity of chaos. AAAS Publications, Washington, DC

8. He Y, Wu M, She JH (2006) An improved global asymptotic stability criterion for delayed cellular neural networks. IEEE Trans Neural Netw 17:250–252

9. Kennedy J, Eberhart R (1995) Particle swarm optimization. IEEE Int Conf 4:1942–1948

10. Liang X, Tan W, Wan Z, Yang D (2008) A novel criterion for glob al asymptotic stability of cellular neu-ral networks with time delays. In: 7th World congress on intelligent control and automation WCICA’08, June 2008, pp 4430–4433

11. Lin YJ, Hou CL, Su TJ (2009) Cellular neural network for noise cancellation of gray image based on hybrid linear matrix inequality and particle swarm optimization. In: 2009 International conference on new trends in information and service science, Beijing, China, 30 June–2 July 2009, pp 613–617 12. Lopez PD, Vilarino LD, Cabello D (2000) Design of multilayer discrete time cellular neural networks

for image processing tasks based on genetic algorithms. IEEE Int Symp Circuits Syst 4:133–136 13. Matei RP (2000) Image processing using hysteretic cellular neural networks. IEEE Int Symp Circuits

14. Ming L, Min L (2004) The robustness design of templates of CNN for detecting inner corners of objects in gray-scale images. IEEE Int Conf Commun Circuits Syst 2:1090–1093

15. Shi Y, Eberhart R (1998) A modified particle optimizer. In: Proceedings of the 1998 IEEE world congress on computational intelligence, May 1998, pp 69–73

16. Shi Y, Eberhart R (1998) Parameter selection in particle swarm optimization. In: Proceedings of the 7th international conference on evolutionary programming VII. Lecture notes in computer science, pp 591–600

17. Singh V (2004) Global asymptotic stability of cellular neural networks with unequal delays: LMI approach. Electron Lett 40:548–549

18. Zhang H, Wang Z (2007) Global asymptotic stability of delayed cellular neural networks. IEEE Trans Neural Netw 18:947–950