具有相互影響之遞迴式自我演化類神經模糊系統及其應用

(1)

國

立

交

通

大

學

電機與控制工程學系

博

士

論

文

具有相互影響之遞迴式自我演化類神經模糊系統及其應用

A Novel Recurrent Self-evolving Neural Fuzzy System and

Its Applications

研究生：林洋印

指導教授：張志永教授

(2)

具有相互影響之遞迴式自我演化類神經模糊系統及其應用

A novel Recurrent Self-evolving Neural Fuzzy System and

Its Applications

研究生：林洋印 Student：Yang-Yin Lin

指導教授：張志永博士 Advisor：Dr. Jyh-Yeong Chang

國立交通大學

電機與控制工程學系

博士論文

A Dissertation

Submitted to Department of Electrical and Control Engineer College of Electrical Engineering

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Doctor of Philosophy

in

Electrical and Control Engineering April 2013

Hsinchu, Taiwan, Republic of China

(3)

具有相互影響之遞迴式自我演化類神經模糊系

統及其應用

研究生：林洋印

指導教授：張志永博士

國立交通大學電機與控制工程學系﹙研究所﹚博士班

摘要

本篇論文提出以相互影響地遞迴式架構為基礎之類神經模糊系統及其應用於動態系統辨識。而此論文主要分成四大部份, 第二部份詳細介紹相互影響地遞迴式架構與類神經模糊系統作結合，並且在後鑑部份使用輸入變數的非線性組合，不同於傳統 Takagi-Sugeno-Kang （TSK）架構，它是利用函數展開的方式，能在高維度的輸入空間中提供良好的非線性決策能力，因此，可使網路輸出更具體且更逼近目標輸出。在架構學習上，使用線上學習模糊分群法，而此演算法能有效地處理時變特徵問題。在參數學習上，後鑑部參數的更新是由可變動維度之卡爾曼濾波器演算法調整,可具有高精密學習的性能。前鑑部及遞迴式參數則利用梯度下降法去做參數更新的動作。在第三部份中，我們提出區間第二類型模糊邏輯系統結合發展出的遞迴式網路，即相互影響地遞迴式區間第二類型類神經模糊系統。區間第二類型模糊邏輯系統具有良好的雜訊容忍度，能直接處理規則的不確定性，這是第一類型模糊邏輯系統所不能達到的。在架構學習上，相互影響之遞迴式區間第二類型類神經模糊系統最初不包含任何規則，所有規則的產生是由線上第二類型模糊分群所取得。在參數學習上，後鑑部參數的更新是由排序後規則之卡爾曼濾波器演算法調整以改善系統的性能。前鑑部及遞迴參數的更新由梯度下降法做調整。在此，我們提出參數消除的方法針對無效的遞迴參數，因規則數太大，會產生許多遞迴參數，我們將冗餘的遞迴參數刪除，來減少網路的計算量。模擬結果顯示出針對

(4)

ii

動態系統在無喧雜的狀況下，所提出的相互影響之遞迴式模糊類神經網路具有優越的性能。最後，我們將與其他方法做比較，證實所提出的架構是卓越的。

(5)

A Novel Recurrent Self-evolving Neural Fuzzy

System and Its Applications

Student：Yang-Yin Lin Advisor：Dr. Jyh-Yeong Chang

Department of Electrical and Control Engineering

National Chiao-Tung University

ABSTRACT

This dissertation mainly describes two different kinds of recurrent neural fuzzy systems, involving a novel recurrent self-evolving fuzzy neural network for identification and prediction of time-varying plants and a novel recurrent interval type-2 neural fuzzy system with self-evolving structure and parameter for dynamic system processing under noise-free and noise environment. For the first kind, we describe a novel recurrent self-evolving neural fuzzy system, namely an interactively recurrent self-evolving fuzzy neural network (IRSFNN). The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN can be chosen by a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNN’s learning starts with an empty rule base and all of rules are generated and learned online through a simultaneous structure and parameter learning. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN

(6)

iv

recurrent FNNs. The proposed model obtains enhanced performance results. For the second kind, we introduce a mutually recurrent interval type-2 neural fuzzy system (MRIT2NFS) with self-evolving structure and parameters for system identification under both noise-free and noisy environments. The MRIT2NFS employs interval type-2 set in the premise clause in order to enhance noise tolerance of system. The consequent part of each recurrent fuzzy rule is defined using the Takagi-Sugeno-Kang (TSK) type with interval weights. The structure learning of MRIT2NFS uses on-line type-2 fuzzy clustering to determine number of fuzzy rules. For parameter learning, the consequent part parameters are tuned by rule-ordered Kalman filter algorithm to reinforce parameter learning ability. The type-2 fuzzy sets in the antecedent and weights representing the mutual feedback are learned by gradient descent algorithm. After the training, a weight-elimination scheme eliminates feedback connections that do not have much effect on the network behavior. This method can efficiently remove redundant recurrence weights. Simulation results show that the MRIT2NFS produces smaller root mean squared errors using the same number of iterations.

(7)

Acknowledgment

在這研究期間裡，首先要感謝我的博士班指導教授張志永博士和林進

燈博士，在二位教授豐富的學識、殷勤的教導及嚴謹的督促下，使我學習

到許多的寶貴知識及在面對事情中應有的處理態度、方法，並且在研究與

投稿論文的過程中，二位教授有許多深入的見解及看法且對於斟酌字句、

思慮周延，更是我該學習的目標。師恩浩蕩，指導提攜，銘感於心。

由衷感謝口試委員李祖聖教授、蘇順豐教授、洪宗貝教授、楊谷洋教授、

及林正堅教授給予許多寶貴的建議與指正，使得這篇論文更加完整。同時

要感謝交大腦科學中心全體同學，在研究的過程中不斷的互相砥礪及討

論。感謝過去 97 級中興電機所的同學，在這過程中，一起分享不同的經驗。

感謝一路走來陪伴身邊的所有朋友，使得我的研究生涯變得多采多姿。

特別要感謝我的父親、母親、姐姐，在這段日子中不斷的給予支持及鼓

勵，讓我能夠專心於研究的工作並完成博士學位。最後誠摯地以本論文研

究成果獻給我的師長、父母、家人及所有的朋友們。

林洋印

一百零二年四月二十三日

(8)

vi

List of Tables

Table 2.1 PERFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR DYNAMIC SYSTEM IDENTIFICATION IN EXAMPLE 1 ... 21 Table 2.2: PERFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR DYNAMIC SYSTEM IDENTIFICATION IN EXAMPLE 2. ... 23 Table 2.3: PEFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR CHAOTIC SEQUENCE PREDICTION IN EXAMPLE 3. ... 25 Table 2.4: PERFORMANCE OF IRSFNN AND OTHER MODELS FOR MACKEY-GLASS CHAOTIC SEQUENCE PREDICTION PROBLEM IN EXAMPLE 4. ... 27 Table 2.5: PERFORMANCE OF IRSFNN AND OTHER MODELS FOR BOX-JENKINS PREDICTION IN EXAMPLE 5. ... 30 Table 3.1: PERFORMANCE OF MRIT2NFS AND OTHER RECURRENT MODELS FOR SISO PLANT IDENTIFICATION IN EXAMPLE 1... 52 Table 3.2: INFLUENCE OF f and _th  ON THE PERFORMANCE OF MRIT2NFS WITH

0.5

  IN EXAMPLE 1. ... 53 Table 3.3: INFLUENCE OF  ON THE PERFORMANCE OF MRIT2NFS WITH f_th 0.02. .. 55 Table 3.4: PERFORMANCE OF MRIT2NFS AND OTHER FEEDFORWARD AND RECURRENT MODELS WITH DIFFERENT NOISE LEVEL IN EXAMPLE 1 ... 56 Table 3.5: PERFORMANCE OF MRIT2NFS AND OTHER RECURRENT MODELS FOR SISO PLANT IDENTIFICATION IN EXAMPLE 2 ... 58 Table 3.6: INFLUENCE OF f_th and  ON THE PERFORMANCE OF AN MRIT2NFS WITH

0.5

  ... ... 59 Table 3.7: PERFORMANCE OF MRIT2NFS AND OTHER FEEDFORWARD AND RECURRENT MODELS WITH DIFFERENT NOISE LEVEL IN EXAMPLE 2. ... 60 Table 3.8: PERFORMANCE OF MRIT2NFS AND OTHER FEEDFORWARD AND RECURRENT MODELS IN EXAMPLE 3. ... 62 Table 3.9: PERFORMANCE OF MRIT2NFS AND OTHER FEEDFORWARD AND RECURRENT MODELS WITH DIFFERENT NOISE LEVEL IN EXAMPLE 3. ... 62 Table 3.10: PERFORMANCE OF MRIT2NFS AND OTHER RECURRENT MODELS FOR MIMO

PLANT IDENTIFICATION IN EXAMPLE 4 ... 65 Table 3.11: PERFORMANCE OF MRIT2NFS AND OTHER MODELS WITH DIFFERENT NOISE

LEVELS IN EXAMPLE 4. ... 65 Table 3.12: PERFORMANCE OF MRIT2NFS- AND OTHER MODELS FOR MODELING OF

(10)

viii

List of Figures

Figure 2.1: Structure of FLANN. ... 9

Figure 2.2: Structure of the proposed IRSFNN model ... 10

Figure 2.3: Flowchart of the structure and parameter learning of the IRSFNN. ... 14

Figure 2.4: Outputs of the dynamic plant, IRSFNN, RSEFNN_LF, and TRFN. ... 21

Figure 2.5: Test errors between the IRSFNN-FuL and actual plant outputs. ... 21

Figure 2.6: Outputs of the dynamic plant and IRSFNN-FuL. ... 23

Figure 2.7: Test errors between the IRSFNN-FuL and actual plant outputs ... 23

Figure 2.8: Results of the phase plot for the chaotic system RSEFNN_FL, TRFN and IRSFNN-FuL ... 24

Figure 2.9: Test result of chaotic series prediction using IRSFNN-FuL. ... 26

Figure 2.10: Prediction errors between the IRSFNN-FuL and actual outputs. ... 27

Figure 2.11: Test result of Box-Jenkins series using IRSFNN-FuL with three rules. ... 29

Figure 2.12: Prediction errors between the IRSFNN-FuL and actual outputs. ... 30

Figure 3.1: Structure of a Type-2 FLS. ... 31

Figure 3.2: The different forms of type-2 fuzzy sets. ... 32

Figure 3.3: The plots of complexity comparison with five type-reducers ... 33

Figure 3.4: The proposed six-layer MRIT2NFS structure. ... 35

Figure 3.5: An interval type-2 fuzzy set with Gaussian shape whose center (mean) is not known with certainty (the mean has an uncertainty). ... 36

Figure 3.6: Outputs of the dynamic plant and MRIT2NFS in example 1. ... 51

Figure 3.7: Test errors between the MRIT2NFS and actual plant outputs ... 51

Figure 3.8: Outputs of the dynamic plant and MRIT2NFS in example 2 ... 57

Figure 3.9: Test errors between the MRIT2NFS and actual plant outputs ... 57

Figure 3.10: Results of the phase plot for the chaotic system (O) and MRIT2NFS (X) ... 61

Figure 3.11: Output of the MIMO system and MRIT2NFS in Example 4. (a) Output yp1. (b) Output yp2 ... 64

(11)

Chapter 1 Introduction

Dynamic systems depend on past inputs, past outputs, or both, and identification and modeling of such systems is not as straightforward as that for static / algebraic systems. For dynamic system processing, practical problems are encountered in a variety of areas, such as control, pattern recognition, time series prediction, and signal processing. Recently, the combination of recurrent structures and fuzzy neural networks has become popular to identify and recognize temporal behaviors [1–16, 43–44]. Therefore, recurrent structures enable effectively address temporal sequences responding to memory information from prior system states.

In contrast with pure feed-forward FNN, we have to know the number of lagged inputs and outputs in advance, and feed these lagged values as feed-forward FNN input. The exact order of dynamic system is usually unknown, and thus, we do not know the number of lagged values to provide. Moreover, the lagged values increase input dimensions and result in a larger network size. Apparently, the use of a feed-forward FNN is unsuitable for constructing dynamic system. Therefore, some recurrent fuzzy neural networks (RFNNs) have already been proposed [1–16, 43–44] for solving the temporal characteristics of dynamic systems, and have been shown to outperform feed-forward FNNs and recurrent neural networks. Recently, a considerable research effort has being devoted toward developing recurrent neural-fuzzy models that are separated into two major categories. One category of recurrent FNNs uses feedback from the network output as the recurrence structure [2–4], [9]. In [2], recurrent self-organizing neural fuzzy inference network (RSONFIN) uses a global feedback structure, where the firing strengths of all rules are summed and fed back as internal network inputs. The other approach of recurrent FNNs uses feedback from internal state variables as its recurrence [10, 12, 14, 44]. In [14], the authors presented a recurrent self-evolving fuzzy

(12)

2

neural network with local feedbacks for dynamic system identification, where the recurrent firing values are influenced by both prior and current values.

One import purpose is to design consequent part of FNN which is able to impact the performance on using different types. Researchers usually use two types of fuzzy if-then rules and fuzzy reasoning employed, i.e., Mamdani-type and TSK-type. For Mamdani-type fuzzy neural networks [2, 6, 17–19], the minimum fuzzy implication is adopted in fuzzy reasoning. For TSK-type fuzzy neural networks [5, 9, 14, 20, 21], the consequent part of each rule is a linear function of input variables. Several studies [14, 20, 21] indicate that the performance of a feedforward TSK-type fuzzy network in network size and learning accuracy is superior to those of Mamdani-type fuzzy networks. A feedforward TSK-type fuzzy network appears to have more free parameters to adjust input space mapping. However, each consequent part of each fuzzy rule in a standard TSK-type fuzzy neural network does not take full advantage of the mapping capabilities of local approximation by rule hyper-planes. Therefore, several studies [22–28] consider trigonometric functions to replace the traditional TSK-type fuzzy reasoning and also obtain the better performance.

In this view, the functional-link neural networks (FLANN) [22, 23] have been proposed using trigonometric functions to construct consequent part. The functional expansion increases the dimensionality of the input vector and thus, creation of nonlinear decision boundaries in the multidimensional space and identification of complex nonlinear functions become simple with this network. It seems to be more efficient, based on these results, to include the functional-link fuzzy rules into the design of recurrent fuzzy network.

With above mentioned motivations, this study presents the combination of a novel recurrent structure and a FLANN to construct the consequent part, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for dynamic system identification and prediction. The proposed IRSFNN contains four major contributions as follows:

(13)

feedback and global feedback. The global feedback in the proposed network means that the necessary information is obtained from the other fuzzy rules. Local source (a rule gets feedback from itself only) is not sufficient to represent the necessary information. Therefore, external (global) backward connection is aim to reimburse the shortcoming of local information and then achieve comprehensive information requirement. (2) Many studies [1–13] have only considered the past states in recurrent structure, which is

insufficient without referring to current states. Previous studies [14, 29] were provided the strong evidences that compatibly use past and current states to be more desirable. Therefore, the proposed recurrent structure depends on current states along with previous states in order to obtain excellent compromise with temporal.

(3) We use the FLNN to replace the traditional TSK-type fuzzy reasoning, and compare their performance. As has explained before, the functional expansion increases the dimensionality of the input pattern and thus, creation of nonlinear decision boundaries in the multidimensional space and identification of complex nonlinear functions become simple with this network.

(4) We use hybrid learning algorithms for parameter learning to reinforce the network performance.

For structure learning, all of the rules and fuzzy sets are generated on-line in an IRSFNN, which helps automate rule generation. We do not need to set any initial IRSFNN structure in advance. The antecedent part and recurrent parameters are learned by gradient descent algorithm. The consequent parameters in an IRSFNN are tuned using a variable-dimensional Kalman filter algorithm. This algorithm handles inputs with variable dimensions, a phenomenon caused by incremental rules during the structure learning process.

All of the recurrent FNNs that we have discussed so far use type-1 fuzzy sets. In recent years, studies on type-2 fuzzy logic systems (FLSs) have drawn much attention [45–49,

(14)

4

involved in the fuzzy rules are type-2 fuzzy sets. We shall refer to such rules as trype-2 fuzzy rules or type-2 rules. The membership values of a type-2 fuzzy set are type-1 fuzzy sets. Type-2 FLSs appear to be more promising than their type-1 counterparts in handling uncertainties, that allow researchers to model and minimize the effect of uncertainties associated with rule-base system, and have already been successfully applied in several areas [49–53, 75–77, 88–91]. The uncertainties in type 2 fuzzy sets can arise from different sources. Four types of uncertainties are described in [46] and [47]. One of them is related to the answers of experts to the same question in different manners. The second type of uncertainty is related to the estimation of the membership function of the same linguistic value by different experts, the third is connected with the noise of measurements that activate type-1 FLS, and the last one is related to the noisy data that are used to tune the parameters of type-1 FLSs. Type-1 fuzzy systems cannot directly model these types of uncertainties. Because the membership functions of type 2 fuzzy systems are themselves fuzzy, they provide a powerful framework to represent and handle such types of uncertainties.

Usually, the T2FNN is computationally more expensive than that of its Type-1 counterpart primarily due to the complexity of type reduction from Type-2 to Type-1. An Interval Type-2 Fuzzy set (IT2FS) is a special case of a general type-2 fuzzy set, which reduces the computational overhead of a general type-2 fuzzy system significantly. For an IT2FS, the membership associated with an element is a sub-interval of [0, 1]. In this dissertation we use the interval type-2 fuzzy modeling to simplify the computational efforts to some extent. In [54]–[59], some interval type-2 FNNs are proposed for designing of interval type-2 FLS. In [29, 60–69, 92], the authors have proposed automatic design of fuzzy rules, which are used in a variety of applications. A self-evolving interval type-2 fuzzy neural network (SEIT2FNN) is proposed in [62], which learns the structure and parameters in an online manner. The premise and consequent parameters in an SEIT2FNN are tuned by gradient descent and rule ordered Kalman filter algorithm, respectively. The performances of

(15)

SEIT2FNN are especially good for time varying systems. Several Interval type-2 FNNs [29, 67–69], which use feedback/recurrent structure are proposed for modeling of dynamic systems. In [67], a recurrent interval type-2 fuzzy neural network (RIT2FNN-A) that uses interval asymmetric type-2 fuzzy sets is proposed. This five-layer FNN uses a four-layer forward network and a feedback layer. In [68], the authors propose an internal/interconnection recurrent type-2 fuzzy neural network (IRT2FNN) structure that is suitable for dealing with time-varying systems. All free parameters of the IRT2FNN are updated via gradient descent algorithm. Moreover, recurrent interval type-2 FNNs with local feedbacks are proposed in [29, 69], where the recurrent property is achieved by locally feeding the firing strength of each rule back to itself. In [29], the consequent part of the recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) is a linear function of current and past outputs and inputs. On the other hand, the consequent part in the RIFNN [69] is of Mamdani type, which is formulated as an interval-valued fuzzy set. In many papers, it has been seen that the Takagi-Sugeno-Kang (TSK) type modeling can do an excellent job of modeling dynamic systems [7, 14, 21, 62, 66, 68].

Here, we propose a Mutually Recurrent Interval Type-2 Neural Fuzzy System (MRIT2NFS) for dynamic system identification. The MRIT2NFS has a self-evolving ability such that it can automatically evolve to acquire the required network structure as well as its parameters based on the training data. Therefore, to start the learning process no pre-assigned network structure is necessary. In this proposed MRIT2NFS, we have three major contributions as follows:

(1) We propose a novel recurrent NFS structure utilizing interval type-2 fuzzy sets. Our network incorporates the advantage of local feedback and effective delivery of information through mutual feedbacks in order to achieve information completeness. In our network, the internal feedback and interaction loops in the antecedent part are

(16)

6

Based on the view of networks, many studies employ external registers to memorize prior states that could cause network’s complexity when number of rules is bigger. Therefore, a internal register is used for reducing network’s complexity. (2) An innovative learning algorithm for the structure and parameters of the system is

suitable for handling time-varying systems, i.e., self-evolving structure and parameter mechanism.

(3) We also propose an interesting scheme to eliminate the less-useful recurrent weights. During the learning process, the MRIT2NFS may generate many recurrent weights when the rule base is bigger. As a result of elimination of the less-useful recurrent weights, our system achieves a significant reduction in both complexity and computational requirements.

The consequent parameters in the MRIT2NFS are tuned by a rule-ordered Kalman filter algorithm. The antecedent parameters and all of the rule recurrent weights are learned by a gradient descent learning algorithm. To demonstrate the performance of MRIT2NFS, several simulations have been conducted. The performance of MRIT2NFS is also compared with that of recurrent type-1 FNNs, feed-forward type-1 FNNs, and other type-2 FNNs.

(17)

Chapter 2 An Interactively Recurrent Self-evolving Fuzzy

Neural Network (IRSFNN)

2.1 Brief Survey of Existing Methods

Recently, considerable research has been devoted toward these developing recurrent fuzzy neural networks, and these networks can be separated into two major categories. One category of recurrent FNNs in studies [1–9, 16], the recurrent structure uses global feedbacks. In [2], a recurrent self-organizing neural fuzzy inference network (RSONFIN) computes the values of the internal feedback variables using all rule firing strengths and the consequent parts are fuzzy sets. The recurrent structure in the RSONFIN just considers past state. For parameter learning, the RSONFIN uses gradient descent algorithm to tune free parameters. The authors in [3] and [4] proposed an output-recurrent fuzzy neural network where the output values are fed back as input values. In [7], the TSK-type recurrent fuzzy network’s structure is similar to an RSONFIN. The recurrent neuron-fuzzy network in [9] feeds back the network output values not only globally to all the rule inputs, but also locally to the consequent part of each rule, in the form of the autoregressive moving average with exogenous inputs model. The recurrent high-order neural network (RHONN) [16] trained with an extended Kalman filter algorithm was proposed for optimal control of nonlinear systems.

The other approach of recurrent FNNs [10–14] uses feedback loops from internal state variables as its recurrence structure. The design of local recurrent structures seems to be simpler than that of global recurrent structures, and also obtains superior performance. In [10] and [11], the recurrent property is achieved by feeding the output of each membership

(18)

8

function back to itself; thus each membership value is only influenced by its previous value. The recurrent property in study [14], a recurrent self-evolving fuzzy neural network with local feedback (RSEFNN-LF), is achieved by locally feeding the output of temporal firing strength back to itself; thus, temporal firing strength is influenced by current and past states.

As mentioned earlier, many researchers frequently use Mandani-type or TSK-type to construct consequent part of fuzzy rules. Many studies indicate that TSK-type fuzzy systems significantly outperform Mandani-type fuzzy systems. However, TSK-type fuzzy neural network does not take full advantage of the mapping capabilities of local approximation by rule hyper-planes. In order to overcome this problem, our proposed model employs the FLANN [22], [23] to strength the mapping ability of input space. Therefore, nonlinear function (trigonometric function) to the consequent part shall be able to effectively discriminate in mapping input space. Previous studies [22–28] indicated that the use of trigonometric function obtains better performances than the use of TSK-type. As a result, in this dissertation the marriage of a novel recurrent structure and functional-link-based NN is a significant research for addressing the temporal problems as demonstrated by every example.

2.2 IRSFNN Structure

This section introduces the structure of the functional link neural network and multiple-input-single-output IRSFNN. The recurrent structure in the IRSFNN uses interaction feedback that has the ability to capture critical information from other rules. The consequent part of each recurrent fuzzy rule is functional link and executes a nonlinear model. Next, we have described the structure of functional link neural network.

2.2.1 Structure of a Functional-link Artificial Neural Network

(19)

structure in which nonlinearity is introduced by enhancing the input pattern with nonlinear functional expansion. Therefore, the FLANN structure considers trigonometric functions. Fig 2.1 shows the structure of FLANN, where each of the input patterns is passed through a functional expansion block yielding a corresponding N -dimensional expanded vector.

Suppose that for the input pattern X is of a two-dimensional input (x x ) the expanded ₁, ₂

inputs are using trigonometric functions to be taken. The expanded input variables can be

denoted as =(1,x ,₁ sin(x₁),cos(x₁),x ,₂ sin(x₂),cos(x₂)) .

Fig. 2.1. Structure of FLANN.

The theory of the FLANN for multidimensional function approximation has been discussed and analyzed below [22, 23]. Let us consider a set of basis functions B{_kΦ( )}A _k__Κ , with the following properties; (1) ₁=1 ; (2) the subset

1 { } k N j k B   B _ is a linearly independent set, that is, if

1 0 N k k k w



 



, then w_k 0 for

k



1,

,

j

; and (3)

1/ 2 2 1 sup_j j _k _A k   _{  } 



 . Next,

{

}

k1 N N k

B





_ is a set of a set of basis functions to be considered, as shown in Fig. 2.1. Hence, output of functional expansion block is composed by (



1,



2, …,



N)BN with the following input–output relationship for the j th output.

ˆ_j ( _j) y  S ; ( ) N j kj k S



w X  



(2.1)

(20)

10

where X  A n , that is, X ( , ,x x₁ ₂ , )x_n T is the input dimension and

1 2

( , ,..., )T j j jN

W w w w is the weight vector associated with the jth output of the FLANN. The vectorS is a matrix of linear outputs of the FLANN, and the output vector is ˆyV, that is,

1 2

ˆ

( ,

ˆ ˆ

,

ˆ

_V

)

T

y



y y

y

. The nonlinear function can be denoted as



 

 

t a n h

 



(2.2) In the IRSFNN model, the corresponding weights of functional link bases do not exist in the initial state, and the amount of the corresponding weights of functional link bases generated by the online learning algorithm is consistent with the number of fuzzy rules. Section 2.3 describes the self-evolving technology.

Fig. 2.2. Structure of the proposed IRSFNN model, where each recurrent fuzzy rule in layer 4 forms a locally and globally recurrent structure and each node in layer 5 combines functional-link-based.

(21)

2.2.2 Structure of IRSFNN

This sub-section describes the IRSFNN model that employs FLANN to the consequent part of the IRSFNN for enhancing network’s performance. Fig 2.2 shows the proposed six-layered IRSFNN structure. The detailed function of each layer is discussed next.

For a clear understanding of the mathematical function of each node, we will describe function relationship between each layer. The net input to the ith node in layer l is represented

as

u

i( )l and the output value is represents as O i( )l

Layer 1 (Input layer): The inputs are crisp values and x( ,x₁ ,x_n) are fed as inputs

to this layer. This is in contrast to feed-forward FNNs where both current and past states are fed as inputs to input layer when such networks are used to model time-varying systems. Weight requiring adjustment in this layer is absent.

Layer 2 (Fuzzification layer): Each node in this layer defines a Gaussian membership

function (MF) and performs a fuzzification operation. For the ith fuzzy set

A

i_j on the input variable

x

j,

j



1,...,

n

, a Gaussian MF is computed by Eq. (1.3)

2 (2) (2) 1 (2) (1) ( ) exp , and 2 i j j i j j i i j j j u m x O u O





 _ _ _       _ _         (2.3)

Layer 3 (Spatial firing layer): Each node in this layer represents one fuzzy rule that

computes the firing strength. Because this layer does not depend on any temporal input, we call this layer “spatial” to distinguish it from the “temporal” firing strength computed in the

next layer. For the obtained spatial firing strength



i , each node performs a fuzzy meet operation on inputs it receives from layer 2 using an algebraic product operation.

(22)

12 (3) (3) (3) (2) 1

, and

n i i j j j j

O

u

O









_(2.4)

where

M

is the total number of rules.

Layer 4 (Temporal firing layer): Each node in this layer is a recurrent rule node, which

formulates an internal feedback (self-loop) and external interaction feedback loop. The ideal of temporal firing strength is extended from the concept of Infinite Impulse Response (IIR) filter that formulates recursive function of prior states and current observation. The output of a recurrent rule node is a temporal firing strength that depends not only on current spatial firing strength but also on the previous temporal firing strength. The temporal firing strength is a linear combination function expressed as

(4) (4) (4) (4) (3) 1

(

q

(

1)) (1

q

)

, and

i ik k i i i i k

O



O

t



u

O









 





_(2.5) that is, 1

( )

(

1)) (1

)

( ),

1,

,

and

1,

,

q q q q i i ik k i k O

t

i

M

q

n



 











 







(2.6) where 1 M q q i ik k









and q q ik ik

C

M





(0C_ikq 1) is the rule interaction weight between

itself and other rules. For the updated recurrent weights, the proposed approach uses a gradient descent algorithm to derive the optimal values. The recurrent weights q

ik

 determine

the compromised ratio between the current and previous inputs to the network outputs.

Layer 5 (Consequent layer): Each node in this layer is an optional node, called a

consequent layer, and can be TSK-type or functional-link-based fuzzy rules. The weight of the

link from a node in layer 4 to one in layer 5 is

a

_iq₀, for q1, ,n_o and

i



1,

,

M

. For the TSK-type IRSFNN, the node output is a linear combination of current input states

1

,

n

x

. The output of TSK-type

q i

v

of the

i

th rule node connecting to the

q

th output variable is computed as follows:

(23)

(5) (4) (4) (1) 0

, and

n q _q i _i _ij _j _j _j j

v

O

a

u

O









_(2.7) where

x

₀



1

.

For the functional-link-based IRSFNN, the output uses a functional expansion as given by the trigonometric polynomial basis function

1 1 1 2 2 2

[ sin(

x



x

) cos(



x

) sin(

x



x

) cos(



x

)]

for two-dimensional input variables. The output of functional-link-based

( )

q i

v t

is expressed by (5) 0

,

3 (

)

t n q _q i _i _ik _k _t _u k

v

O

a



n

n n







  

_(2.8) where



₀



1

.

The coefficient

n

_u denotes lag numbers of system output or control input. If we do not use

extra lagged values (

n

_u =0),



_k



( ,sin(

x

₁



x

₁

),cos(



x

₁

),

, x ,sin(

_n



x

_n

),cos(



x

_n

))

and

k



1,

,

n

_t. The constant

n

_t is an amount of basis expansion according to input variables.

Layer 6 (Output layer): Each node in this layer corresponds to one output variable. For

defuzzification operations, the

q

th output layer node computes the network output variable

q

y

by using the weighted average method.

For the TSK-type IRSFNN, the output can be expressed as

(5) (4) 1 0 (6) 1 (4) 1 1 ( ) ( ) M n M q q i i ij j i i j i q M M q i i i i t a x O O y O O t



        







,

q



1,

,

n

o (2.9)

where v denotes the consequent value and a denotes the parameters.

(24)

14 (4) (5) (6) 1 1 0 (4) 1 1

( )

t n M M q q i i i ik j i i k q M M q i i i i

O O

t

a

y

O

t







    











,

q



1,

,

n

o(2.10)

where

v

denotes the consequent value and

a

denotes the parameters.

2.3 IRSFNN Learning

In this section, two phase learning is used for constructing the IRSFNN. There are no rules in an IRSFNN. All of the recurrent fuzzy rules evolve from the simultaneous structure and parameter learning after receiving each piece of training data. Fig. 2.3 presents flowchart of the IRSFNN’s learning scheme. The parameter learning phase describes the use of a gradient descent algorithm and a variable-dimensional Kalman filter algorithm.

(25)

2.3.1 Structure Learning

The first task in structure learning is to determine whether a new rule should be extracted from the training data and to determine the number of fuzzy sets in the universe of discourse of each input variable because one cluster in the input space corresponds to one potential

fuzzy rule, in which

m

i_j represents the mean and



ij represents the variance of that cluster.

The spatial firing strength



i in (2.4) is used to determine whether a new rule should be generated. The first incoming data point x is used to generate the first fuzzy rule, and the mean and width of the fuzzy membership functions associated with this rule are set as:

1

j j

m



x

and



1j





fixed ,

j



1,

,

n

(2.11)

where



_fixed is a predefined value (we use



_fixed=0.3 in this paper) that determines the width of the memberships associated with a new rule. For subsequent new incoming data x(t) we find 1 ( )

arg max

_ci

( )

i M t

I

f t

 



(2.12) where M(t) is the number of existing rules at time t. If

f

_cI

( )

t



f

_th(

f

_th is a pre-specified threshold), then a new fuzzy rule is generated and M(t+1)=M(t)+1. In this approach, if the present data do not match well according to the existing rules, then a new rule is evolved. We also use the same procedure to assign the mean of fuzzy sets as we have done for first rule. For a new rule, the mean and width of corresponding fuzzy sets are defined as

( ) 1 M t j j

m





x

and



M tj ( ) 1



xj mIj  _{ } _ ,

j



1,

,

n

(2.13) where  is an overlap coefficient. Eq. (1.13) indicates that the initial width is equal to the Euclidean distance between current input data

x

and the center of the best matching rule for this data point times an overlapping parameter



. In this study



is set to 0.5, so that the width of new fuzzy set is half of the Euclidean distance from the best matching center, and a

(26)

16 suitable overlap between adjacent rules is realized.

2.3.2 Parameter Learning

In addition with the structure, all free parameters in an IRSFNN are also learned, including those newly generated and previously existing. For clarification, we consider the single-output case and define the objective to minimize the error function as

2

1 ( )

( )

2

q d

E





_

y t



y t



_

(2.14)

where y t_q( ) represents the IRSFNN output and

y t

_d

( )

represents the desired output. Parameters in the consequent part of the TSK-type IRSFNN are learned based on the variable-dimensional Kalman filter algorithm as discussed in [14]. According to [14], Eq. (2.10) of a functional-link-based IRSFNN can be re-written as

( )

T q FuL FuL

y





t

a

(2.15) where 1 1 1 ( 1) 1 1 0 0 1 1 1 1

( )

, ,

( )

t t t t t n n q q q q n M T M M FuL M M n M M n q q q q i i i i i i i i

t







        































(2.16) and ( 1) 1 10

,

1

,

0

,

t t t T _M _n q q q q FuL _n _M _Mn

a





_

a



_



   (2.17) and 0 1 1 1 1

[ , , ,

] [1, ,sin(

),cos(

), , ,sin(

),cos(

)]

t

n

x

n

x

n

x

n

 







(2.18)

The consequent parameter vector aFuL is updated by executing the following

(27)

(

1)

( )

(

1)

(

1)(

(

1)

(

1)

( )),

( )

(

1)

(

1) ( )

1 (

1)

( )

(

1) ( )

(

1)

FuL FuL

FuL FuL _FuL _d _FuL FuL

T FuL T FuL

a

t

a

t

S t

t

y t

t

a

t

S t

t

S t

t

S t

t





 



 



 









 

















(2.19)

where



is a forgetting factor and lies in [0,1] (



=0.99995 in this paper). Once a new rule is generated, the dimension of the vectors

a

_FuL ,



_FuL , and the matrix S increases

accordingly. When a new rule evolves at time t+1, the new vector



_FuL

(

t



1)

becomes

1 1 1 ( 1) ( 1) 1 1 1 1 0 0 1 1 1 1

( )

(

1)

, ,

( )

t t t t t n n q q q q n M T M M FuL M M n M M n q q q q i i i i i i i i

t







           

































(2.20)

An IRSFNN augments a_FuL( )t and

S t

( )

on the right-hand side of Eq. (2.17) as follows:

( 1) ( 1) 1 _ , ₍ ₁₎₀

,

₍ ₁₎ t t T M n q q

FuL new FuL _M _M _n

a





_

a

_

a

_



_



    (2.21) and ( 1)( 1) ( 1)( 1)

( )

[ ( )

]

M nt M nt new

S

t



block diag S t C I

 

     (2.22) where Cis a large positive constant (we use C=10).

This paper uses resetting operations to keep S bounded and to avoid divergence problems. After a period of training, the matrix S is re-set as

C I



. Simulation results in Section 2.4 show that the learning of the IRSFNN achieves good training and test performance. A gradient descent algorithm tunes the antecedent parameters of the IRSFNN. This gradient descent algorithm is performed once for each piece of incoming datum.

By using a gradient descent algorithm for the updated recurrent weights, we have

(

1)

( )

q q ik ik q ik

E

t









 





(2.23) where



is the learning rate and

(28)

18 1

(

) (

(

1)

( )) /

( )

q q i q q q ik q i ik M q q i q q d i q k i i

y

E

y

v

y

t



















_



_



 













 



(2.24)

The antecedent part of parameter m is updated as i_j

(

1)

( )

i i j j i j

E

m t

m





 





(2.25) where 2 1

(

)

2(

)

(

)

(1

)

(

)

( )

i q i q _i j i q i i i j q i j j q i i q q i j j q d M i i q _j i i

y

E

m

y

m

v

y

x

m

y

t





























 











 

 



(2.26)

The antecedent part of parameter



ij is updated as

(

1)

( )

i i j j i j

E

t









 





(2.27) where 2 3 1

(

)

2(

)

(

)

(1

)

(

)

( )

i q i q _i j i q i i i j q i j j q i i q q i j j q d M i i q _j i i

y

E

y

v

y

x

m

y

t













 













_



_



 











 

 



(2.28)

2.4 Simulation Results

This section presents five examples to assess the performance of IRSFNN and MRIT2NFS. These simulation studies include two types of dynamic system problems (Examples 1–2) and three types of prediction problems (Examples 3–5). These examples are

(29)

also used to compare the performance of the IRSFNN with those of existing recurrent FNNs. For dynamic system identification, the recurrent structure in the proposed approach shows the advantages, as listed in Tables 2.1–2.5.

2.4.1. Example 1 (Dynamic System Identification)

This example uses an IRSFNN to identify a nonlinear dynamic system, which is a nonlinear plant with multiple time delays that has been studied in [7]. The dynamic system is described by the following difference equation

( 1) ( ( ), ( 1), ( 2), ( ), ( 1)) p p p p y t  f y t y t y t u t u t (2.29) where 1 2 3 5 3 4 1 2 3 4 5 2 2 2 3

(

1)

( ,

,

)

1 x x x x x

x

f x x x x x

x







(2.30)

The dynamic system output depends on three previous outputs and two previous inputs. In

this study, only two current values, u t and ( ) y t_p( ), are fed as input to the IRSFNN input layer. Here, we do not use extra lagged values (n =0) in the consequent part. The desired _u

output of the IRSFNN is y t_d( 1). In the training procedure of the IRSFNN, we follow the same computational protocols as in [7] and [14], i.e., we use only 10 epochs, with 900 time steps in each epoch. In each epoch, the first 350 inputs are random values uniformly distributed over [-2, 2] and the remaining 550 training inputs are generated from a sinusoid,

1.05sin(



t

/ 45)

.

We follow this strategy for an online training process because a similar procedure was followed in [14], where the total number of online training time steps is 9000. The structure

learning threshold fth decides the number of rules to be generated. After training, three rules

(30)

20

root–mean-squared error (RMSE) of training data. The testing input signal u t is guided by ( )

sin( ), 250 25 1.0, 250 t<500 ( ) 1.0, 500 t<750 0.3sin( ) 0.1sin( ) 25 32 +0.6sin( ), 10 t t u t t t t           750 t<1000               (2.31)

Fig. 2.4 shows a comparison of the actual output with the output produced by the IRSFNN for the test input. Fig. 2.5 shows the error difference between the actual plant output and the IRSFNN. Figs. 2.4–2.5 show a very good match, suggesting that IRSFNN architecture combined with the proposed system identification scheme adequately identifies the dynamic system with feedback.

Table 2.1 shows the performance of the IRSFNN compared with the other recurrent networks, including a recurrent self-organizing neural fuzzy inference network (RSONFIN) [2], a wavelet recurrent fuzzy neural network (WRFNN) [11], a TSK-type recurrent fuzzy network (TRFN) [7], a HO-RNFS [6], and a recurrent self-evolving fuzzy neural network with local feedback (RSEFNN-LF) [14].

The consequent part in the RSEFNN-LF is composed by a first-order TSK-type that performs a linear combination of input variables. As in the IRSFNN, all these networks use the same information including the number of input variables, training data, test data, and training epochs. For a fair comparison, the total number of parameters of the IRSFNN is kept similar to that of the compared networks. The result indicates that the IRSFNN achieves better identification than the other recurrent networks.

(31)

0 100 200 300 400 500 600 700 800 900 1000 -1.5 -1 -0.5 0 0.5 1 Time Step O u tp u t ideal output IRSFNN-FuL RSEFNN-LF TRFN

Fig. 2.4. Outputs of the dynamic plant (blue line), IRSFNN (red line), RSEFNN_LF (green line), and TRFN (black line) in Example 1.

0 100 200 300 400 500 600 700 800 900 1000 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Time Step E rr o r

Fig. 2.5. Test errors between the MRIT2FNN and actual plant outputs.

Fig. 5. Test errors between the MRIT2FNN and actual plant outputs

TABLE 2.1 PERFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR DYNAMIC SYSTEM IDENTIFICATION IN EXAMPLE 1

Models RSONFIN [2] WRFNN [11] HO-RNFS [6] TRFN [7] RSEFNN-LF [14] IRSFNN (TSK) IRSFNN (FuL) Rules 4 5 3 3 4 3 3 Number of parameters 36 55 45 33 32 30 42 Training RMSE 0.025 0.064 0.054 0.032 0.020 0.015 0.011 Test RMSE 0.078 0.098 0.082 0.047 0.040 0.036 0.031

(32)

22

2.4.2. Example 2 (Dynamic System Identification)

This example considers the use of the IRSFNN for dynamic system identification with longer input delays that is described by

2

1 1 1

( 1) 0.72 ( ) 0.025 ( 1) ( 1) 0.01 (

2) 0.2 ( 3)

p p p

y t

 

y t



y t



u t

 

u t

 

u t



(2.32) This plant is the same as the one used in [7]. This plant output depends on four previous inputs and two previous outputs. As shown in Example 1, the current variables u t( )_and

( )

p

y t

are fed as inputs to the IRSFNN input layer. In this example, we do not use extra lagged values (n =0) in the consequent part. The training data and time steps are the same as _u

those used in Example 1. When the structure learning threshold f is set to 0.05, three rules _th

are generated. The test signal used in Example 1 is also adopted here to assess the identified system. Fig. 2.6 shows the outputs of the plant and the IRSFNN for these test inputs. Fig. 2.7 shows the test error between the outputs of the IRSFNN and the desired plant. Table 1.2 shows the number of rules, total number of parameters, and training and test RMSEs of the IRSFNN. The performance of the IRSFNN is compared with that of recurrent models, including an RSONFIN [2], a TRFN [7], a WRFNN [11], and an RSEFNN-LF [14]. These models use identical numbers of input variables, training data, test data, and training epochs as designed by the IRSFNN. For a fair comparison, the numbers of parameters in the IRSFNN have been kept similar to those in these compared models.

Apparently in Table 2.2, the RSEFNN-LF only uses local source, which is not enough to capture critical information for the system, thus the test error of the IRSFNN_TSK is lower than that of the RSEFNN-LF, even using fewer rules. Here, we also investigate the performance comparison of the IRSFNN-TSK and the IRSFNN-FuL, and results show that the IFSFNN-FuL achieves better performance. Finally, the results show that the test RMSEs of the IRSFNN-FuL and IRSFNN-TSK are smaller than those of the other networks.

(33)

0 100 200 300 400 500 600 700 800 900 1000 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Time Step O u tp u t ideal output IRSFNN-FuL RSEFNN-LF TRFN

Fig. 2.6. Outputs of the dynamic plant (blue line) and IRSFNN-FuL (red line) in Example 2.

0 100 200 300 400 500 600 700 800 900 1000 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Time Step E rr o r

Fig. 2.7. Test errors between the IRSFNN-FuL and actual plant outputs.

TABLE 2.2 PERFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR DYNAMIC SYSTEM IDENTIFICATION IN EXAMPLE 2

Models RSONFIN [2] WRFNN [11] TRFN [7] RSEFNN-LF [14] IRSFNN (TSK) IRSFNN (FuL) Rules 6 5 3 4 3 2 Number of Parameters 36 55 33 30 30 26 Training RMSE 0.03 0.057 0.007 0.016 0.014 0.011 Test RMSE 0.06 0.083 0.031 0.028 0.026 0.022

(34)

24

2.4.3. Example 3 (Chaotic Series Prediction)

As introduced in [30], the IRSFNN is applied to predict the Henon chaotic sequence of a dynamic system with one delay and two sensitive parameters generated by the following equation:

2

(

1)

-

( )

(

1) 1.0

p p p

y t

 

P y



t

 

Q y t

 

(2.33) Eq. (2.33), with P=1.4 and Q=0.3, produces a chaotic attractor. The initial states

[y_p(1),y_p(0)] [0.4, 0.4] generate 2000 patterns, with the 1000 patterns used for training and the remaining 1000 patterns used for testing. In this example, we do not use extra lagged values (n =0) in the consequent part. The training procedure uses the plant output _u y t_p( 1) as the desired output y t_d( 1). The system has a single output so that only output variable

( )

p

y t is fed as input to the IRSFNN. The training epoch in the IRSFNN is set to 90. The

structure learning threshold

f

_th is set to 0.2 and number of rules generated is 5 after the training procedure. Fig. 2.8 shows the phase plot of the actual and IRSFNN predicted results for the test patterns

Fig. 2.8. Results of the phase plot for the chaotic system(blue), RSEFNN_FL (green), TRFN(black) and IRSFNN-FuL(red).

Table 2.3 includes the network size, parameter numbers, and training and test RMSEs of the IRSFNN. The TSK-type IRSFNN and functional-link-based IRSFNN both use four rules. We

(35)

find that the latter achieves greater learning performance. In a meaningful comparison, the number of parameters of the IRSFNN must be similar to that of compared models. The compared recurrent models include a recurrent FNN [12], a wavelet recurrent FNN [13], a TSK-type recurrent fuzzy network [9], and a recurrent self-evolving FNN with local feedback [16], where the locally recurrent is a simple structure but its performance is superior to that of other compared models. The training epochs, training data and test data of the compared models are identical to the conditions of the IRSFNN. Table 2.3 shows that the IRSFNN exhibits the best performance by using an interactively recurrent structure.

2.4.4. Example 4 (Mackey-Glass Chaotic Series Prediction)

The time series prediction problem used in this example is the well-known Mackey-Glass chaotic series. The Mackey-Glass time series is generated from the delay differential equation: 10

( )

0.2 (

)

0.1 ( )

1 (

)

dx t

x t

dt

x

t













(2.34)

where  =17 and the initial value is given as x(0)=1.2. Four past values are used to predict ( )

x t , and the input-output data format is [ (x t24), (x t18), (x t12), (x t6); ( )]x t . As

discussed in [3, 9, 14, 16], 1000 patterns are generated from t=124 to t=1123, with the first TABLE 2.3 PEFORMANCE OF IRSFNN AND OTHER RECURRENT MODELS FOR CHAOTIC

SEQUENCE PREDICTION IN EXAMPLE 3

Models RFNN [10] WRFNN [11] TRFN-S [7] RSEFNN-LF [14] IRSFNN (TSK) IRSFNN (FuL) Rules 15 7 6 9 4 3 Number of Parameters 60 70 66 45 32 40 Training RMSE 0.463 0.191 0.028 0.032 0.017 0.016 Test RMSE 0.469 0.188 0.027 0.023 0.015 0.014

(36)

26

500 patterns being used for training and the remaining 500 for testing. The IRSFNN’s training

epoch is set to 500, and its structure threshold f_th is set to 0.001. After 500 epochs of training procedure, seven rules are generated. The four input dimensions contain 28 fuzzy sets. Fig. 2.9 displays the prediction results of the IRSFNN-FuL, and Fig. 2.10 shows the prediction error between desired output and the IRSFNN. Figs. 2.9–2.10 show an excellent match, suggesting that the proposed scheme in the network indicates and excellent ability to predict the Mackey-Glass time series. Table 2.4 shows the performance, including rules, a total number of parameters, and training and test RMSE for the TSK-type and Functional-ink-based IRSFNNs.

Table 2.4 lists the performance comparison of the IRSFNN with recently developed fuzzy systems designed by particle swarm algorithms or neural learning ([1], [7], [12], [14], and [31–37]). Both local linear wavelet NN (LLWNN) [36] and fuzzy wavelet NN (FWNN) [37] employ the wavelet neural network in the consequent part, which has the ability to localize in both time and frequent space. The compared models with particle swarm algorithm were proposed as a clustering-aided simplex particle swarm optimization (CSPSO) [34], a self-evolving evolutionary learning algorithm (SEELA) designed for neural fuzzy inference system [32] and FLNFN-CCPSO [35]. The FLNFN-CCPSO also uses the function-link-based neural network to the consequent part in the FLNFN-CCPSO.

(37)

0 50 100 150 200 250 300 350 400 450 500 -8 -6 -4 -2 0 2 4 6 8x 10 -4 Time Step E rr o r

Fig. 2.10. Prediction errors between the IRSFNN-FuL and actual outputs in Example 4.

The proposed models, especially the IRSFNN-FuL, show superior performance to compared models. Although the performance of the IRSFNN-TSK is similar to that of

TABLE 2.4 PERFORMANCE OF IRSFNN AND OTHER MODELS FOR MACKEY-GLASS CHAOTIC SEQUENCE PREDICTION

PROBLEM IN EXAMPLE 4 Models Rules Number of

parameters Train RMSE Test RMSE D-FNN [12] 10 100 – 0.0082 G-FNN [1] 10 90 – 0.0056 Recurrent ANFIS [31] – – – 0.0013 SEELA [32] 9 198 0.0067 0.0068 SuPFuNIS [33] 10 94 – 0.0057 TRFN-S [7] 5 95 – 0.0124 CSPSO [34] 10 104 – 0.0064 FLNFN-CCPSO [35] – – 0.0083 0.0084 LLWNN+Hybrid [36] – 110 0.0033 0.0036 FWNN [37] 16 128 0.0023 0.0023 RSEFNN-LF [14] 9 94 0.0032 0.0031 IRSFNN (TSK) 5 90 0.0040 0.0039 IRSFNN (FuL) 4 100 0.0002 0.0002

具有相互影響之遞迴式自我演化類神經模糊系統及其應用

國

立

交

通

大

學

電機與控制工程學系

博

士

論

文