Motivation

Fuzzy systems and neural networks have attracted the growing interest of researchers in various scientific and engineering areas. The number and variety of applications of fuzzy systems and neural networks [1-6] have been increasing, ranging from consumer products and industrial process control to medical instrumentation, information systems, and decision analysis.

Fuzzy systems are structured numerical estimators. They start from highly formalized insights about the structure of categories found in the real world and then articulate fuzzy IF-THEN rules as a kind of expert knowledge. Fuzzy systems combine fuzzy sets with fuzzy rules to produce overall complex nonlinear behavior.

Neural networks, on the other hand, are trainable dynamical systems whose learning, noise-tolerance, and generalization abilities grow out of their connectionist structures, their dynamics, and their distributed data representation. Neural networks have a large number of highly interconnected processing elements (nodes) which demonstrate the ability to learn and generalize from training patterns or data; these simple processing elements also collectively produce complex nonlinear behavior.

The performance of fuzzy systems critically depends on the input and output membership functions, the fuzzy rules, and the fuzzy inference mechanism. On the other hand, the performance of neural networks depends on the computational function of the neurons in the network, the structure and topology of the network, and the learning rule or the update rule of the connecting weights. The advantages and

disadvantages of fuzzy systems and neural networks are summarized as follows [7]:

The advantages of the fuzzy systems are:

• capacity to represent inherent uncertainties of the human knowledge with linguistic variables;

• simple interaction of the expert of the domain with the engineer designer of the system;

• easy interpretation of the results, because of the natural rules representation;

• easy extension of the base of knowledge through the addition of new rules;

• robustness in relation of the possible disturbances in the system.

The disadvantages of the fuzzy systems are:

• incapable to generalize, or either, it only answers to what is written in its rule base;

• not robust in relation the topological changes of the system, such changes would demand alterations in the rule base;

• depends on the existence of a expert to determine the inference logical rules;

The advantages of the neural networks are:

• learning capacity;

• generalization capacity;

• robustness in relation to disturbances.

The disadvantages of the neural networks are:

• impossible interpretation of the functionality;

• difficulty in determining the number of layers and number of neurons.

The hybrid neuro-fuzzy systems [8-34] possess the advantages of both neural networks (e.g. learning abilities, optimization abilities, and connectionist structures) and fuzzy systems (e.g. humanlike IF-THEN rules thinking and ease of incorporating expert knowledge). In this way, we can bring the low-level learning and computational power of neural networks into fuzzy systems and also high-level,

humanlike IF-THEN rule thinking and reasoning of fuzzy systems into neural networks.

There are several different ways to develop hybrid neuro-fuzzy systems;

therefore, being a recent research subject, each researcher has defined its own particular models. These models are similar in its essence, but they present basic differences. The most popular neuro-fuzzy architectures include: 1) Fuzzy Adaptive Learning Control Network [8][20][21][29][35]; 2) Adaptive-Network-Based Fuzzy Inference System [24]; 3) Self-Constructing Neural Fuzzy Inference Network [25];

and 4) Functional-Link-Based Neuro-Fuzzy Network [32][33]. The advantages of a combination of neural networks and fuzzy inference systems are obvious [8][34-36].

Fusion of artificial neural networks and fuzzy inference systems have attracted the growing interest of researchers in various scientific and engineering areas due to the growing need of adaptive intelligent systems to solve the real world problems [8][9][19][20][24][25][30][33-38].

No matter which neuro-fuzzy architecture is chosen, training of the parameters is the main problem in designing a neuro-fuzzy system. Backpropagation (BP) [20][24][25][32][35][38][39] training is commonly adopted to solve this problem. It is a powerful training technique that can be applied to networks with a forward structure.

Since the steepest descent approach is used in BP training to minimize the error function, the algorithms may reach the local minima very quickly and never find the global solution. The aforementioned disadvantages lead to suboptimal performance, even for a favorable neuro-fuzzy system topology. Therefore, technologies that can be used to train the system parameters and find the global solution while optimizing the overall structure are required.

Figure 1.1 sketches a rough taxonomy of global optimization methods [40].

Generally, optimization algorithms can be divided in two basic classes: deterministic

and probabilistic algorithms. Deterministic algorithms are most often used if a clear relation between the characteristics of the possible solutions and their utility for a given problem exists. Then, the search space can efficiently be explored using for example a divide and conquer scheme. If the relation between a solution candidate and its “fitness” are not so obvious or too complicated, or the dimensionality of the search space is very high, it becomes harder to solve a problem deterministically.

Trying it would possible result in exhaustive enumeration of the search space, which is not feasible even for relatively small problems. Then, probabilistic algorithms come into play.

An especially relevant family of probabilistic algorithms is the Monte Carlo-based approaches. They trade in guaranteed correctness of the solution for a shorter runtime. This does not mean that the results obtained using them are incorrect - they may just not be the global optima. An important class of probabilistic Monte Carlo metaheuristics is evolutionary computation (EC). It encompasses all algorithms that are based on a set of multiple solution candidates (called population) which are iteratively refined. This field of optimization is also a class of soft computing as well as a part of the artificial intelligence area. Some of its most important members are evolutionary algorithms (EAs) and swarm intelligence (SI).

The particle swarm optimization (PSO) developed by Kennedy and Eberhart in 1995 [41-43], is a relatively new technique. Although PSO shares many similarities with evolutionary computation techniques, the standard PSO does not use evolution operators such as crossover and mutation. PSO emulates the swarm behavior of insects, animals herding, birds flocking, and fish schooling where these swarms search for food in a collaborative manner. Each member in the swarm adapts its search patterns by learning from its own experience and other members’ experiences.

State Space

Figure 1.1: The taxonomy of global optimization algorithms.

During the past several years, PSO has been successfully applied to a diverse set of optimization problems, such as multidimensional optimization problems [44], multi-objective optimization problems [45-47], classification problems [48][49], and feedforward neural network design [39][50-53]. Aggregation chart for applications of the PSO over different years is shown in Figure 1.2 [54].

Figure 1.2: Aggregation chart for applications of the PSO over different years.

In this dissertation, we proposed the novel learning algorithms embedded with particle swarm optimizer for the neural fuzzy system in both classification and nonlinear system control applications.