Organization of Dissertation - 內嵌粒子群優化學習演算法之類神經模糊系統及其應用

Chapter 1 Introduction

1.3 Organization of Dissertation

The overall objective of this dissertation is to develop the novel learning algorithms embedded with particle swarm optimizer for the neuro-fuzzy systems. The proposed learning algorithms are suitable for any neuro-fuzzy architecture. In this

research, we take the functional-link-based neuro-fuzzy network (FLNFN) model for example to demonstrate the performance of the proposed learning algorithms.

Organization and objectives of each chapter in this dissertation are as follows.

In Chapter 2, we describe the structure of FLNFN model. The FLNFN model is based on our laboratory’s previous research [32]. Each fuzzy rule corresponds to a sub-FLNN [80-82] comprising a functional expansion of input variables. The functional link neural network (FLNN) is a single layer neural structure capable of forming arbitrarily complex decision regions by generating nonlinear decision boundaries with nonlinear functional expansion. Therefore, the consequent part of the FLNFN model is a nonlinear combination of input variables, which differs from the other existing models [20][24][25].

In Chapter 3, we propose an efficient immune-based particle swarm optimization (IPSO) algorithm for neuro-fuzzy classifiers to solve the skin color detection problem.

The proposed IPSO algorithm combines the immune algorithm (IA) and PSO to perform parameter learning. The IA uses the clonal selection principle, such that antibodies between others of high similar degree are affected, and these antibodies, after the process, will have higher quality, accelerating the search and increasing the global search capacity. On the other hand, we employed the advantages of PSO to improve the mutation mechanism of IA. Simulations have conducted to show the performance and applicability of the proposed method.

In Chapter 4, we present an evolutionary neural fuzzy classifier, designed using the neural fuzzy system (NFS) and a new evolutionary learning algorithm. This new evolutionary learning algorithm is based on a hybrid of bacterial foraging optimization (BFO) and PSO. It is thus called bacterial foraging particle swarm optimization (BFPSO). The proposed BFPSO method performs local search through the chemotactic movement operation of bacterial foraging whereas the global search

over the entire search space is accomplished by a particle swarm operator. The proposed NFS with BFPSO learning algorithm (NFS-BFPSO) is adopted in several classification applications. Experimental results have demonstrated that the proposed NFS-BFPSO method can outperform other methods.

In Chapter 5, we present an evolutionary NFS for nonlinear system control. A supervised learning algorithm, which consists of structure learning and parameter learning, is presented. The structure learning depends on the entropy measure to determine the number of fuzzy rules. The parameter learning, based on the PSO algorithm, can adjust the shape of the membership function and the corresponding weighting of the FLNN. The distance-based mutation operator, which strongly encourages a global search giving the particles more chance of converging to the global optimum, is introduced. The simulation results have shown the proposed method can improve the searching ability and is very suitable for the nonlinear system control applications.

In Chapter 6, we compare the performance of the proposed learning algorithms using skin color detection problem. In addition, a brief discussion of the proposed learning methods is also made.

Finally, Chapter 7 draws conclusions and future works.

Chapter 2 Structure of the Functional-Link-Based Neuro-Fuzzy Network

In the field of artificial intelligence, neural networks are essentially low-level computational structures and algorithms that offer good performance when they deal with sensory data. However, it is difficult to understand the meaning of each neuron and each weight in the networks. Generally, fuzzy systems are easy to appreciate because they use linguistic terms and IF-THEN rules. However, they lack the learning capacity to fine-tune fuzzy rules and membership functions. Therefore, neuro-fuzzy networks combine the benefits of neural networks and fuzzy systems to solve many engineering problems.

In [83], the definition of hybrid neuro-fuzzy system is as follows: “A hybrid neuro-fuzzy system is a fuzzy system that uses a learning algorithm based on gradients or inspired by the neural networks theory (heuristic learning strategies) to determine its parameters (fuzzy sets and fuzzy rules) through the patterns processing (input and output)”. In other words, neuro-fuzzy networks bring the low-level learning and computational power of neural networks into fuzzy systems and give the high-level human-like thinking and reasoning of fuzzy systems to neural networks.

Recently, neuro-fuzzy networks have become popular topics of research. The advantages of a combination of neural networks and fuzzy inference systems are obvious [8][34-36]. They not only have attracted considerable attention due to their diverse applications in fields such as pattern recognition, image processing, prediction, and control, but they can also handle imprecise information through linguistic

expressions. The most popular neuro-fuzzy architectures include: 1) Fuzzy Adaptive Learning Control Network (FALCON) [8][20][21][29][35]; 2) Adaptive-Network-Based Fuzzy Inference System (ANFIS) [24]; 3) Self-Constructing Neural Fuzzy Inference Network (SONFIN) [25]; and 4) Functional-Link-Based Neuro-Fuzzy Network (FLNFN) [32][33].

In this dissertation, the selected NFS model is based on our laboratory’s previous research [32][33], called FLNFN. Figure 2.1 presents the structure of the FLNFN model, which combines a neuro-fuzzy network with a FLNN [80-82]. The FLNN [81][84] is a single layer neural structure capable of forming arbitrarily complex decision regions by generating nonlinear decision boundaries with nonlinear functional expansion. Moreover, the FLNN was conveniently used for function approximation and pattern classification with faster convergence rate and less computational loading than a multilayer neural network. In the selected FLNFN model, each fuzzy rule that corresponds to a FLNN consists of a functional expansion of input variables, which differs from the other existing models [20][24][25].

The FLNFN model realizes a fuzzy IF-THEN rule in the following form.

Rule j : linguistic term of the precondition part with a Gaussian membership function; N is the number of input variables; w is the link weight of the local output; _kj _k is the basis trigonometric function of input variables; M is the number of basis functions, and Rule j is the j^th fuzzy rule.

ˆ

x x ˆ

₂

ˆy3

ˆy2

ˆy1

ˆ

x x ˆ

₂

Figure 2.1: Structure of the selected neuro-fuzzy system model.

The operation functions of the nodes in each layer of the FLNFN model are now described. In the following description, u denotes the output of a node in the ^{( )}^l l ^th layer.

Layer 1 (Input node): No computation is performed in this layer. Each node in

this layer is an input node, which corresponds to one input variable, and only transmits input values to the next layer directly:

i x

u⁽¹⁾  ˆ (2.2)

Layer 2 (Membership function node): Nodes in this layer correspond to a single

linguistic label of input variables in layer 1. Therefore, the calculated membership value specifies the degree to which an input value belongs to a fuzzy set in layer 2.

The implemented Gaussian membership function in layer 2 is

(1) 2 Layer 3 (Rule Node): Nodes in this layer represent the preconditioned part of a fuzzy logic rule. They receive one-dimensional membership degrees of the associated rule from the nodes of a set in layer 2. Here, the product operator described above is adopted to perform the IF-condition matching of the fuzzy rules. As a result, the output function of each inference node is



uij⁽²⁾ of a rule node represents the firing strength of its corresponding rule.

Layer 4 (Consequent Node): Nodes in this layer are called consequent nodes.

The input to a node in layer 4 is the output from layer 3, and the other inputs are nonlinear combinations of input variables from a FLNN, as shown in Figure 2.1. For such a node, expansion of input variables. Considering the computational efficiency, the functional expansion uses a trigonometric polynomial basis function, given by

     1, , , , , 2 3 4 5 6  xˆ1, sin (xˆ1), cos (xˆ1), , xˆ2 sin (xˆ2), cos (xˆ2) for the

two-dimensional input variablesx xˆ ˆ1, 2. Therefore, M is the number of basis functions, M  3 N, where N is the number of input variables. Moreover, the output nodes of FLNN depend on the number of fuzzy rules of the FLNFN model.

Layer 5 (Output Node): Each node in this layer corresponds to a single output

variable. The node integrates all of the actions recommended by layers 3 and 4 and acts as a center of area (COA) defuzzifier with



As described above, the number of tuning parameters for the FLNFN model is known to be (2 3 ) P   , where N R N, R , and P denote the number of inputs, existing rules, and outputs, respectively.

Chapter 3 Immune Algorithm Embedded with Particle Swarm Optimizer for

Neuro-Fuzzy Classifier and Its Applications

Skin color detection is the process of finding skin-colored pixels and regions in an image or a video. This process is typically used as a preprocessing step to find regions that potentially have human faces and limbs in images. Several computer vision approaches have been developed for skin color detection. A skin color detector typically transforms a given pixel into an appropriate color space and then use a skin color classifier to label the pixel whether it is a skin or a non-skin pixel. A skin color classifier defines a decision boundary of the skin color class in the color space based on a training database of skin-colored pixels.

This chapter presents the efficient immune-based particle swarm optimization (IPSO) for neuro-fuzzy classifiers to solve the skin color detection problem. The proposed IPSO algorithm combines the immune algorithm (IA) and particle swarm optimization (PSO) to perform parameter learning. The IA uses the clonal selection principle to affect antibodies between others of high similar degree, and these antibodies, after the process, will be of higher quality, accelerating the search, and increasing the global search capacity. The PSO algorithm, proposed by Kennedy and Eberhart [41-43], has proved to be very effective for solving global optimization. It is not only a recently invented high-performance optimizer that is easy to understand

and implement, but it also requires little computational bookkeeping and generally only a few lines of code [85]. In order to avoid trapping in a local optimal solution and to ensure the search capability of a near global optimal solution, mutation plays an important role in IPSO. Therefore, we employ the advantages of PSO to improve mutation mechanism of IA. The proposed method can improve the searching ability and greatly increase the converging speed that we can observe in the simulations.

3.1 Basic Concepts of the Artificial Immune System

The biological immune system is successful at protecting living bodies from the invasion of various foreign substances, such as viruses, bacteria, and other parasites (called antigens), and eliminating debris and malfunctioning cells. Over the last few years, a growing number of computer scientists have carefully studied the success of this competent natural mechanism and proposed computer immune models, named artificial immune systems (AIS), for solving various problems [86-94]. AIS aim at using ideas gleaned from immunology in order to develop adaptive systems capable of performing a wide range of tasks in various areas of research.

In this research, we review the clonal selection concept, together with the affinity maturation process, and demonstrate that these biological principles can lead to the development of powerful computational tools. The algorithm to be presented focuses on a systemic view of the immune system and does not take into account cell-cell interactions. It is not our concern to model exactly any phenomenon, but to show that some basic immune principles can help us not only to better understand the immune system itself, but also to solve complex engineering tasks.

3.2 Clonal Selection Theory

Any molecule that can be recognized by the adaptive immune system is known as an antigen (Ag). When an animal is exposed to an Ag, some subpopulation of its bone-marrow-derived cells (B lymphocytes) responds by producing antibodies (Ab’s).

Ab’s are molecules attached primarily to the surface of B cells whose aim is to recognize and bind to Ag’s. Each B cell secretes a single type of antibody (Ab), which is relatively specific for the Ag. By binding to these Ab’s (cell receptors) and with a second signal from accessory cells, such as the T-helper cell, the Ag stimulates the B cell to proliferate (divide) and mature into terminal (non-dividing) Ab secreting cells, called plasma cells. The process of cell division (mitosis) generates a clone, i.e., a cell or set of cells that are the progenies of a single cell. While plasma cells are the most active Ab secretors, large B lymphocytes, which divide rapidly, also secrete Ab’s, albeit at a lower rate. On the other hand, T cells play a central role in the regulation of the B cell response and are preeminent in cell mediated immune responses, but will not be explicitly accounted for the development of our model.

Lymphocytes, in addition to proliferating and/or differentiating into plasma cells, can differentiate into long-lived B memory cells. Memory cells circulate through the blood, lymph and tissues, and when exposed to a second antigenic stimulus commence to differentiate into large lymphocytes capable of producing high affinity antibodies, pre-selected for the specific antigen that had stimulated the primary response [95]. In this study, we treat the long-lived B memory cells as the better antibodies by elitism selection. Figure 3.1 depicts the clonal selection principle [95].

The main features of the clonal selection theory [96][97] that will be explored in this study are:

• Proliferation and differentiation on stimulation of cells with Ag’s;

• Generation of new random genetic changes, subsequently expressed as diverse Ab patterns, by a form of accelerated somatic mutation (a process called affinity maturation);

• Elimination of newly differentiated lymphocytes carrying low affinity antigenic receptors.

Figure 3.1: The clonal selection principle.

3.3 The Efficient Immune-Based PSO Learning Algorithm

This section describes the efficient immune-based PSO (IPSO) learning algorithm for use in the neuro-fuzzy classifier. Analogous to the biological immune system, the proposed algorithm has the capability of seeking feasible solutions while maintaining diversity. The proposed IPSO combines the immune algorithm (IA) and particle swarm optimization (PSO) to perform parameter learning. The IA uses the

clonal selection principle to accelerate the search and increase global search capacity.

The PSO algorithm has proved to be very effective for solving global optimization. It is not only a recently invented high-performance optimizer that is very easy to understand and implement, but it also requires little computational bookkeeping and generally only a few lines of code. In order to avoid trapping in a local optimal solution and to ensure the search capability of a near global optimal solution, mutation plays an important role in IPSO. Moreover, the PSO adopted in evolution algorithm yields high diversity to increase the global search capacity, as well as the mutation scheme. Therefore, we employed the advantages of PSO to improve the mutation mechanism of IA. A detailed IPSO of the neuro-fuzzy classifier is presented in Figure 3.2. The whole learning process is described step-by-step below.

3.3.1 Code fuzzy rule into antibody

The coding step is concerned with the membership functions and the corresponding parameters of the consequent part of a fuzzy rule that represent Ab’s suitable for IPSO. This step codes a rule of a neuro-fuzzy classifier into an Ab. Figure 3.3 shows an example of a neuro-fuzzy classifier coded into an Ab (i.e. an Ab represents a rule set), where i and j represent the i^th dimension and the j^th rule, respectively. In this research, a Gaussian membership function is used with variables representing the mean and standard deviation of the membership function. Each fuzzy rule has the form in Figure 2.1, where m_ij and _ij represent a Gaussian membership function with mean and standard deviation of the i^th dimension and j^th rule node and w_ij represents the corresponding parameters of consequent part.

Figure 3.2: Flowchart of the proposed IPSO algorithm.

Figure 3.3: Coding a neuro-fuzzy classifier into an antibody in the IPSO method.

3.3.2 Determine the initial parameters by self-clustering algorithm Before the IPSO method is designed, the initial Ab’s in the populations are generated according to the initial parameters of the antecedent part and the consequent part. In this study, the initial parameters of a neuro-fuzzy classifier were computed by the self-clustering algorithm (SCA) method [52][98][99]. That is, we used SCA method to determine the initial mean and standard deviation of the antecedent part.

On the other hand, the initial link weight of the consequent part is a random number in the range of 0 to 1.

SCA is a distance-based connectionist clustering algorithm. In any cluster, the maximum distance between an example point and the cluster center is less than a threshold value. This clustering algorithm sets clustering parameters and affects the number of clusters to be estimated. In the clustering process, the data examples come from a data stream. The clustering process starts with an empty set of clusters. The clusters will be updated and changed depending on the position of the current example in the input space.

3.3.3 Produce initial population

In the immune system, the Ab’s are produced in order to cope with the Ag’s. In other words, the Ag’s are recognized by a few of high affinity Ab’s (i.e. the Ag’s are optimal solutions). The first initial Ab utilizing a real variable string is generated by

SCA, and the other Ab’s of population are generated based on the first initial Ab by adding some random value.

3.3.4 Calculate affinity values

For the large number of various Ag’s, the immune system has to recognize them for their posterior influence. In biological immune system, affinity refers to the binding strength between a single antigenic determinants and an individual antibody-combining site. The process of recognizing Ag’s is to search for Ab’s with the maximum affinity with Ag’s. Moreover, every Ab in the population is applied to problem solving, and the affinity value is a performance measure of an Ab which is obtained according to the error function. In this study, the affinity value is designed according to the follow formulation:

 

ND represents the number of the training data. In the problems, the higher affinity refers to the better Ab.

3.3.5 Production of sub-antibodies

In this step, we will generate several neighborhoods to maintain solution variation. This strategy can prevent the search process from becoming premature. We can generate several clones for each Ab on feasible space by Eqs. (3.2), (3.3) and (3.4).

Each Ab regarded as parent while the clones regarded as children (sub-antibodies). In other words, children regarded as several neighborhoods of near parent.

mean: clones children[ _i_{_}_c] antibody parent[ _i] (3.2)

deviation: clones children[ _i_{_}_c] antibody parent[ _i] (3.3) weight : clones children[ _i_{_}_c] antibody parent[ _i]  (3.4)

where parent_i represents the i^th Ab from the Ab population; children_{i c}_{_} represents clones number c from the i^th Ab;  and  are parameters that control the distance between parent. In this scheme,  and  are important parameters. The large values lead to the speed of convergence slowly and the search of optimal solution difficulty, whereas the small values lead to fall in a local optimal solution easily. Therefore, the selection of the  and  will critically affect the learning results, and their values will be based on practical experimentation or on trial-and-error tests.

3.3.6 Mutation of sub-antibodies based on PSO

In order to avoid trapping in a local optimal solution and to ensure the search capability of near global optimal solution, mutation plays an important role in IPSO.

Moreover, the PSO adopted in evolution algorithm yields high diversity to increase the global search capacity, as well as the mutation step. Hence, we employed the advantages of PSO to improve mutation mechanism. Through the mutation step, only one best child can survive to replace its parent and enter the next generation.

PSO is a recently invented high-performance optimizer that is very easy to understand and implement. Each particle has a velocity vector v_i and a position vector x_i to represent a possible solution. In this research, the velocity for each particle is updated by Eq. (1.3). The parameter w(0, 1] is the inertia of the particle,

在文檔中內嵌粒子群優化學習演算法之類神經模糊系統及其應用 (頁 24-0)