Literature Survey

Recently, neuro-fuzzy networks [1]-[20] provide the advantages of both neural networks and fuzzy systems, unlike pure neural networks or fuzzy systems alone. Neuro-fuzzy networks

(NFN) bring the low-level learning and computational power of neural networks into fuzzy systems and give the high-level human-like thinking and reasoning of fuzzy systems to neural networks.

Two typical types of neuro-fuzzy networks are the Mamdani-type and the Takagi-Sugeno-Kang (TSK)-type. For Mamdani-type neuro-fuzzy networks [4]-[6], the minimum fuzzy implication is adopted in fuzzy reasoning. For TSK-type neuro-fuzzy networks (TSK-type NFN) [7]-[10], the consequence part of each rule is a linear combination of input variables. Many researchers [9]-[10] have shown that TSK-type neuro-fuzzy networks offer better network size and learning accuracy than Mamdani-type neuro-fuzzy networks. In the typical TSK-type neuro-fuzzy network, which is a linear polynomial of input variables, the model output is approximated locally by the rule hyper-planes. Nevertheless, the traditional TSK-type neuro-fuzzy network does not take full advantage of the mapping capabilities that may be offered by the consequent part.

Introducing a nonlinear function, especially a neural structure, to the consequent part of the fuzzy rules has yielded the NARA [21] and the CANFIS [22] models. These models [21]-[22] apply multilayer neural networks to the consequent part of the fuzzy rules. Although the interpretability of the model is reduced, the representational capability of the model is markedly improved. However, the multilayer neural network has such disadvantages as slower convergence and greater computational complexity. Therefore, this dissertation uses the functional link neural network (FLNN) [23]-[25] to the consequent part of the fuzzy rules, called a functional-link-based neuro-fuzzy network (FLNFN). The consequent part of the proposed FLNFN model is a nonlinear combination of input variables, which differs from the other existing models [5], [9]-[10]. The FLNN is a single layer neural structure capable of forming arbitrarily complex decision regions by generating nonlinear decision boundaries with nonlinear functional expansion. The FLNN [26] was conveniently used for function approximation and pattern classification with faster convergence rate and less computational

loading than a multilayer neural network. Moreover, using the functional expansion can effectively increase the dimensionality of the input vector, so the hyper-planes generated by the FLNN will provide a good discrimination capability in input data space.

In addition, training of the parameters is the main problem in designing a neuro-fuzzy network. Backpropagation (BP) training is commonly adopted to solve this problem. It is a powerful training technique that can be applied to networks with a forward structure. Since the steepest descent approach is used in BP training to minimize the error function, the algorithms may reach the local minima very quickly and never find the global solution. The aforementioned disadvantages lead to suboptimal performance, even for a favorable neuro-fuzzy network topology. Therefore, technologies, that can be used to train the system parameters and find the global solution while optimizing the overall structure, are required.

Recent development in genetic algorithms (GAs) has provided a method for neuro-fuzzy system design. Genetic fuzzy systems (GFSs) [27]-[31] hybridize the approximate reasoning of fuzzy systems with the learning capability of genetic algorithms. GAs represent highly effective techniques for evaluating system parameters and finding global solutions while optimizing the overall structure. Thus, many researchers have developed GAs to implement fuzzy systems and neuro-fuzzy systems in order to automate the determination of structures and parameters [32]-[52].

Carse et al. [32] presented a GA-based approach to employ variable length rule sets and simultaneously evolves fuzzy membership functions and relations called Pittsburgh-style fuzzy classifier system. Herrera et al. [33] proposed a genetic algorithm-based tuning approach for the parameters of membership functions used to define fuzzy rules. This approach relied on a set of input-output training data and minimized a squared-error function defined in terms of the training data. Homaifar and McCormick [34] presented a method that simultaneously found the consequents of fuzzy rules and the center points of triangular membership functions in the antecedent using genetic algorithms. Velasco [35] described a

Michigan approach which generates a special place where rules can be tested to avoid the use of bad rules for online genetic learning. Ishibuchi et al. [36] applied a Michigan-style genetic fuzzy system to automatically generate fuzzy IF-THEN rules for designing compact fuzzy rule-based classification systems. The genetic learning process proposed is based on the iterative rule learning approach and it can automatically design fuzzy rule-based systems by Cordon et al. [37]. A GA-based learning algorithm called structural learning algorithm in a vague environment (SLAVE) was proposed in [38]. SLAVE used an iterative approach to include more information in the process of learning one individual rule. Furthermore, a very interesting algorithm was proposed by Russo in [39] which attempted to combine all good features of fuzzy systems, neural networks and genetic algorithm for fuzzy model derivation from input-output data. Chung et al. [40] adopted both neural networks and GAs to automatically determine the parameters of fuzzy logic systems. They utilized a feedforward neural network for realizing the basic elements and functions of a fuzzy controller. In [41], a hybrid of evolution strategies and simulated annealing algorithms is employed to optimize membership function parameters and rule numbers which are combined with genetic parameters.

Three main strategies, including Pittsburgh-type, Michigan-type, and the iterative rule learning genetic fuzzy systems, focus on generating and learning fuzzy rules in genetic fuzzy systems. First, the Pittsburgh-type genetic fuzzy system [42] was characterized by using a fuzzy system as an individual in genetic operators. Second, the Michigan-type genetic fuzzy system was used for generating fuzzy rules in [43], where each fuzzy rule was treated as an individual. Thus, the rule generation methods in [43] were referred to as fuzzy classifier systems. Third, the iterative rule learning genetic fuzzy system [44] was adopted to search one adequate rule set for each iteration of the learning process. Moreover, Ishibuchi et al. [45]-[48]

proposed genetic algorithms for constructing a fuzzy system consisting of a small number of linguistic rules. Mitra et al. [49]-[52] presented some approaches that exploit the benefits of

soft computation tools for rule generation.

In the aforementioned literatures, it has been fully demonstrated that GAs are very powerful in searching for the true profile. However, the search is extremely time-consuming, which is one of the basic disadvantages of all GAs. Although the convergence in some special cases can be improved by hybridizing GAs with some local search algorithms, it is achieved at the expense of the versatility and simplicity of the algorithm. Similar to GAs, DE [53]-[55]

also belongs to the broad class of evolutionary algorithms, but DE has many advantages such as the strong search ability and the fast convergence ability over GAs or any other traditional optimization approach, especially for real valued problems [55]. Therefore, we propose a rule-based symbiotic modified differential evolution (RSMODE) for the proposed FLNFN model. The RSMODE is to adjust the system parameters and find the global solution while optimizing the overall structure.