A TSK-Type-Based Self-Evolving Compensatory
Interval Type-2 Fuzzy Neural Network
(TSCIT2FNN) and Its Applications
Yang-Yin Lin, Jyh-Yeong Chang, Member, IEEE, and Chin-Teng Lin, Fellow, IEEE
Abstract—In this paper, a Takagi–Sugeno–Kang (TSK)-type-based self-evolving compensatory interval type-2 fuzzy neural network (FNN) (TSCIT2FNN) is proposed for system modeling and noise cancellation problems. A TSCIT2FNN uses type-2 fuzzy sets in an FNN in order to handle the uncertainties associated with information or data in the knowledge base. The antecedent part of each compensatory fuzzy rule is an interval type-2 fuzzy set in the TSCIT2FNN, where compensatory-based fuzzy reasoning uses adaptive fuzzy operation of a neural fuzzy system to make the fuzzy logic system effective and adaptive, and the consequent part is of the TSK type. The TSK-type consequent part is a linear combination of exogenous input variables. Initially, the rule base in the TSCIT2FNN is empty. All rules are derived according to online type-2 fuzzy clustering. For parameter learning, the consequent part parameters are tuned by a variable-expansive Kalman filter algorithm to the reinforce parameter learning abil-ity. The antecedent type-2 fuzzy sets and compensatory weights are learned by a gradient descent algorithm to improve the learning performance. The performance of TSCIT2FNN for the identification is validated and compared with several type-1 and type-2 FNNs. Simulation results show that our approach produces smaller root-mean-square errors and converges more quickly.
Index Terms—Compensatory operation, fuzzy identification, online fuzzy clustering, type-2 fuzzy systems.
I. INTRODUCTION
R
ECENTLY, fuzzy neural networks (FNNs) have become popular in applications in control, identification, predic-tion, pattern recognipredic-tion, and bioengineering. FNNs inherit their learning ability from neural networks and their infer-ence technology from fuzzy systems and are used for solving the aforementioned characteristic behaviors [1]–[12], such as in the control of robot manipulators [4], temperature control [5], pattern classification [6], ventricular premature contraction Manuscript received March 23, 2012; revised July 10, 2012, September 15, 2012, and November 27, 2012; accepted February 14, 2013. Date of publication February 22, 2013; date of current version July 18, 2013. This work was supported in part by the Aiming for the Top University Plan of National Chiao Tung University, and in part by the Ministry of Education, Taiwan, under Con-tract 101W963. Further support came from the University System of Taiwan-University of California, San Diego, International Center of Excellence in Advanced Bioengineering sponsored by the Taiwan National Science Council International Research-Intensive Centers of Excellence Program under Grant NSC-100-2911-I-009-101.Y.-Y. Lin and J.-Y. Chang are with the Institute of Electrical Control Engineering, National Chiao Tung University, Hsinchu 30010, Taiwan (e-mail: [email protected]).
C.-T. Lin is with the Institute of Electrical Control Engineering, National Chiao Tung University, Hsinchu 30010, Taiwan, and also with the Brain Research Center, National Chiao Tung University, Hsinchu 300, Taiwan.
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIE.2013.2248332
(VPC) detection [7], energy conversion [8], and hardware im-plementation [9]. FNNs are an effective tool for dealing with complex nonlinear processes. Some well-known feedforward FNNs include an adaptive neuro-fuzzy inference system [2], an online self-constructing neural fuzzy inference network (SONFIN) [3], and fuzzy wavelet neural networks (FWNNs) [10]–[12].
Moreover, the aforementioned FNNs use only a static fuzzy inference mechanism, such as a Min or Max operator, to achieve fuzzy reasoning. There are two disadvantages to using fixed fuzzy operators. The first is that there are not enough free variables to adjust the fuzzy operator, and the second is that fixed fuzzy operators cannot optimize the fuzzy logic reasoning. In [13], Zimmermann and Zysno first defined the essence of a compensatory operator in order to the overcome aforementioned disadvantages. The enhanced compensatory operation was presented according to whether a pessimistic or optimistic operation was to be taken [14]. Many studies [15]–[19] have dealt with the optimization of the fuzzy mem-bership function (MF), as well as defuzzification via parameter learning algorithms. In [20]–[24], compensatory attempts were widely used in a variety of FNNs. As there are fewer type-2 fuzzy logic systems (FLSs) using a compensatory fuzzy infer-ence mechanism, we will extend the compensatory mechanism to type-2 FLSs in this study.
The type-2 fuzzy set is an extension of the type-1 fuzzy set, as first introduced by Zadeh [1]. Theoretical developments in type-2 sets have been widely reported in the literatures [25]–[29]. Indeed, studies on type-2 FLSs [30]–[33] have drawn great attention because type-2 fuzzy sets seem to be a more effective approach for handling uncertainties in the rule base associated with information and data, with successful applica-tions in a variety of areas, such as control [34]–[36], medical applications [37], and noise tolerance analysis [38]. However, generalized type-2 FLSs are more complex than type-1 FLSs. The theory of interval type-2 fuzzy sets has been proposed [26] in order to improve the computational complexity and load. Many researchers have considered using interval type-2 fuzzy systems. Therefore, the design of an interval type-2 FNN has drawn considerable efforts. In [39]–[42] and [65], the authors proposed the parameter learning of an interval type-2 fuzzy system using Gaussian primary MFs with uncertain means. In [39], the optimal training of an interval type-2 FNN using the gradient descent algorithm is discussed. In [43], a hybrid learning algorithm for interval type-2 FNNs is presented. Recently, some interval type-2 FNNs have been proposed for the automatic design of type-2 FLSs [44]–[51]. In [44], a 0278-0046/$31.00 © 2013 IEEE
self-evolving interval type-2 FNN (SEIT2FNN) model uses interval type-2 sets with uncertain means in the antecedent parts and interval type-1 Takagi–Sugeno–Kang (TSK) fuzzy sets in the consequent parts. In a further study described in [46], the type-2 FNN uses two forms of interval type-2 fuzzy sets, one with uncertain means but fixed standard deviations (STDs) and the other with uncertain STDs but a fixed mean, and compares their performances. Some type-2 FNNs [44], [46], [51] are trained using a Kalman filter algorithm to improve the learning performance. In [48], the author proposed a novel type-2 TSK-based fuzzy neural structure (FNS) for the identification and control of dynamic plants. The design of two-axis motion control using an interval type-2 fuzzy set was proposed in [49], and the structure of a discrete interval type-2 fuzzy system by fuzzy c-means (FCM) clustering has been proposed in [50].
In industry, most plants are susceptible to uncertainties in internal and external disturbances and focus on time-varying technique. Therefore, the purpose of this study is to develop an interval type-2 FNN, i.e., a TSK-type-based self-evolving compensatory interval type-2 FNN (TSCIT2FNN), for sys-tem modeling, problem for forecasting, and noise cancella-tion. The self-evolving property means that the TSCIT2FNN can automatically evolve the required network structure (rule number and initial fuzzy set shape) and parameters accord-ing to pieces of trainaccord-ing data. In addition, the self-evolvaccord-ing property enables the efficient handling of the time-varying characteristic problem. In this paper, the major contributions of the TSCIT2FNN are twofold. The first is the proposal of a compensatory operator in the type-2 inference mechanism. To use the compensatory operator, an inference mechanism in the TSCIT2FNN is more flexible for optimizing the fuzzy reasoning via a gradient descent algorithm. For our second contribution, this paper proposes a variable-expansive Kalman filter algorithm for the learning of consequent weights in order to reinforce the learning accuracy.
The rest of this paper is organized as follows. Section II introduces the TSCIT2FNN structure, which combines a com-pensatory operation in the inference mechanism and a crisp TSK function in the consequent part. Section III presents the structure learning of the TSCIT2FNN and its parameter update rules. Section IV describes five simulation examples, including three types of system identification, adaptive noise cancellation (ANC), and time-series prediction. Section V contains our conclusions.
II. TSCIT2FNN STRUCTURE
This section introduces the structure of a multiple-input single-output TSCIT2FNN. Fig. 1 shows the proposed six-layered TSCIT2FNN structure. The premise parts of the TSCIT2FNN utilize the interval type-2 fuzzy sets having un-certain means and fixed STDs, and the consequent part of each compensatory fuzzy rule is of the TSK type and executes a crisp linear model. The proposed TSCIT2FNN realizes the fuzzy if-then rules in the following form:
Rule i :IF x1is ˜Ai1and . . . and xnis ˜Ain 1−γi+γi n THEN y is ai0+ n j=1 aijxj i = 1, . . . , M (1)
Fig. 1. Proposed six-layer TSCIT2FNN structure, where each compensatory rule in layer 4 formulates a compromise between the optimistic and the pessimistic operator and each node in layer 5 performs a linear combination of current input states.
Fig. 2. Interval type-2 fuzzy set with uncertain mean.
where xj, j = 1, . . . , n, denotes input variables and y is the
output variable. ˜Ai
n denotes type-2 MFs used by the Gaussian
MF. γi∈ [0, 1] is a compensatory degree, which can not only adaptively adjust fuzzy MFs but also dynamically optimize the adaptive fuzzy reasoning. ai
0 and aij for the ith rule of the jth
input are TSK-type consequent part parameters, and M is the total number of rules. The detailed mathematical functions of each layer are introduced hereinafter.
Layer 1 (input layer): Each node in this layer is a crisp input variable. Only input variables are fed into this layer. Note that there are no weights to be adjusted in this layer. Layer 2 (MF layer): Each node in this layer utilizes an interval
type-2 MF to perform the fuzzification operation. As stated earlier, the interval type-2 fuzzy set ˜Ai
primary MF having a fixed STD σ and an uncertain mean that takes values in [m1, m2] (see Fig. 2), i.e.,
μA˜i j = exp ⎧ ⎨ ⎩− 1 2 xj− mij σi j 2⎫⎬ ⎭ ≡ N mij, σij; xj , mij∈mij1, mij2. (2) The footprint of uncertainty (FoU) of this MF can be represented as a bounded interval in terms of an upper MF, ¯
μi
j, and a lower MF, μij, where
μij(xj) = ⎧ ⎨ ⎩ N mi j1, σji; xj , xj< mij1 1, mi j1≤ xjmij2 N mi j2, σji; xj , xj> mij2 (3) μi j(xj) = ⎧ ⎨ ⎩ N mi j2, σji; xj , xj≤ mi j1+mij2 2 N mij1, σji; xj , xj> mi j1+m i j2 2 . (4)
That is, the output of each node can be represented as an interval [μi
j, ¯μ i j].
Layer 3 (firing layer): Each node in this layer represents its fuzzy rule and functions. To acquire firing strength Fi,
each node performs a fuzzy meet operation with inputs from layer 2 using an algebraic product operation. The fir-ing strength is an interval type-1 fuzzy set and is computed as follows [27]: Fi= [fi, fi], i = 1, . . . , M (5) fi= n j=1 μij, fi = n j=1 μi j. (6)
Layer 4 (compensatory firing layer): Each node in this layer has its corresponding firing strength, which may be reinforced or weakened. The t-norm fuzzy operation is a pessimistic operation as the min or product operation. The t-conorm fuzzy operation is an optimistic operation as the max operation. Therefore, the use of a compensatory operation can map the pessimistic input x1and the optimistic input
x2to give a relative compromise for superior and inferior
situations. For example, c(x1, x2) = x11−γx γ
2, where γ is
a compensatory degree. Here, the pessimistic value x1
denotes f , and the optimistic value x2denotes f1/n. As a
result, each node in this layer is performed by a pessimistic and optimistic operation, which is called the compensatory operator. Here, the output function of the compensatory operation is denoted as ψi(P, O) = ψi p(t), o(t), ψi(p(t), o(t)) , i = 1, . . . , M (7) where ψi p(t), o(t)= pi(t)1−γi(t)× oi(t)γi(t) = fi(t)1−γi(t)+γi (t)n (8) ψi(p(t), o(t)) = pi(t)1−γi(t)× oi(t)γi(t) = fi(t)1−γi(t)+γi (t)n . (9)
The pessimistic operation of P = [p(t), p(t)] at time t is expressed as
pi(t) = fi(t), pi(t) = fi(t) (10) and the optimistic operation of O = [o(t), o(t)] at time t is expressed as oi(t) = fi(t) 1 n , oi(t) = fi(t) 1 n . (11) Here, γi(t) = c2 i(t)/c2i(t) + d2i(t)∈ [0, 1] is called the
compensatory degree, and ci, di∈ [−1, 1]. The update
parameters ci and di are used to enhance the adaptive
compensatory operator.
Layer 5 (consequent layer): Each node in this layer is called a consequent node and describes a linear model with exogenous inputs. Each compensatory rule node in layer 4 has a corresponding consequent node in layer 5. The output of a consequent node is a linear combination of external input states denoted by
yiTSK= ai0+ n j=1 aijxj = ai0x0+ ainx1+· · · + ainxn (12)
where x0≡ 1, x1, . . . , xn are input variables, and the
consequent part parameters are described by (1).
Layer 6 (output layer): Nodes in this layer correspond to one output linguistic variable. The output function combines the output of layers 3 and 4, and the design factor q enables us to share the upper and the lower values in the final output. Thus, there is no need to apply the Karnik–Mendel (KM) iterative procedure [42] to find the end points. The defuzzified output is given by
y= qy+ (1− q)y= q M i=1 ψiyi TSK M i=1 ψi + (1− q) M i=1 ψiyi TSK M i=1 ψi (13) where ψi and ψi are computed by (8) and (9), and the design factor q can adjust the proportion of the upper and the lower, depending on the certainty of the system.
III. TSCIT2FNN LEARNING
This section explicitly introduces the structure and parameter learning algorithm. Initially, the rule base is empty in the TSCIT2FNN. All of the rules and free parameters are generated and derived based on simultaneous structure and parameter learning.
A. Structure Learning
The online structure learning is generated by a type-2 fuzzy clustering method. A previous study [3] proposed the criterion of type-1 rule generation according to the rule firing strength. This idea protocol is extended to type-2 fuzzy clustering as
a criterion of rule generation in the TSCIT2FNN. The firing strength Fi in (5) is used to determine whether a new rule
should be generated. The type-2 rule firing strength is an interval, with the center of the firing interval (the criterion for rule generation) denoted as
fc= 1 2 fi+ fi . (14)
Next, we discuss the structure-learning algorithm of the TSCIT2FNN. On the arrival of the first incoming data x, the new fuzzy rule is generated immediately. The first uncertain means and fixed width of MFs associated with this rule are set as m1j1, m1j2 = [xj− Δx, xj+ Δx] σ = σfixed, j = 1, . . . , n (15)
where σfixed is a predefined value (we use σfixed= 0.4 in this
paper) that determines the width of the memberships associated with a new rule. Subsequently, for each new incoming data item x(t), we find
I = arg max
1≤i≤M(t)f i
c(t) (16)
where M (t) is the number of existing rules at time t. If fcI(t)≤ fth (fth is a prespecified threshold), then a new fuzzy rule is
generated. The idea is that the present data point is distinct from any of the existing rules, and hence, a new rule is generated. We use the same procedure to assign the uncertain means and width of the generated MF of a new rule. The uncertain means of corresponding type-2 fuzzy sets are defined as
mM (t)+1j1 , mM (t)+1j2
= [xj(t)− Δx, xj(t) + Δx] . (17)
The width of the new rule is defined as follows:
σM (t)+1j = β· xj− mI j1+ mIj2 2 . (18)
Equations (15) and (17) indicate that Δx means the width of the uncertain region where the proper uncertainty region can adapt to the network input range. If the uncertainty associated with the mean is too small, then the type-2 fuzzy sets become similar to type-1 fuzzy sets. On the other hand, if the width of the uncertain region is too large, then the uncertain mean covers most of the input domain. Equation (18) uses the Euclidean distance between the current input data x and the center of the best matching rule for this data point multiplied by an over-lapping parameter β to compute the width of the new type-2 fuzzy sets. In this paper, β is set to 0.5 so that the width of a new type-2 fuzzy set is half the Euclidean distance from the best matching center, and a suitable overlap between adjacent rules is realized. As a result, the overlapping parameter setting is significant for the rule generation procedure.
B. Parameter Learning
Parameter learning is performed simultaneously to the structure-learning process. The gradient descent algorithm and variable-expansive Kalman filter algorithm are performed once each time that incoming data are derived. The antecedent parts and relative compensatory parameters in the TSCIT2FNN are adjusted for incoming data by the gradient descent algorithm, which is suitable for a supervised method. When we consider only the single-output case for clarity, the aim is to minimize the error function
E =1 2[y
(t + 1)− y
d(t + 1)]2 (19)
where y(t + 1) and yd(t + 1) represent the actual network
output and the desired output, respectively. Next, we describe the consequent learning of details in the TSCIT2FNN based on the variable-expansive Kalman filter algorithm. In previous studies [42], the consequent part parameters must be reordered in ascending order, according to the KM iterative procedure, in order to find accurate end points, i.e., L and R points. Moreover, the design factor q enables to efficiently adjust the proportion of the upper and lower in the final output, and thus, we reduce the computational complexity as well as the problem of computation time without the KM iterative procedure in the TSCIT2FNN. The variable-expansive Kalman filter algorithm tunes the consequent part parameters. Therefore, (13) can be reexpressed as y=φ T TSK yTSK (20) where yTSK =a10, . . . , a1n, . . . , aM0 , . . . , aMn T (21) φ T TSK = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ qφ1x0+(1− q)φ 1 x0, . . . , qφ1xn+(1− q)φ 1 xn n+1 .. . qφMx0+(1− q)φ M x0, . . . , qφMxn+(1− q)φ M xn n+1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ T ∈ 1×M(n+1). (22)
Let φ and ¯φ be defined as follows:
φ= ψ M i=1 ψi(t) , φ = ψ M i=1 ψi(t) . (23)
The consequent parameter vector is updated by executing the following variable-expansive Kalman filter algorithm:
yTSK(t + 1)
× yd(t + 1)− φ T TSK(t + 1) yTSK(t) ! S(t + 1) = 1 λ ⎡ ⎣S(t) −S(t) φTSK(t + 1) φ T TSK(t + 1)S(t) λ +φTTSK(t + 1)S(t)φTSK(t + 1) ⎤ ⎦ (24) where λ is a forgetting factor (0 < λ≤ 1). The dimension of the vectors yTSK and φTSK, and the matrix S, increases
whenever a new rule evolves. When a new rule evolves, we express the argument S(t) in the TSCIT2FNN as
S(t) = block diag [S(t), C·I]∈(M +1)(n+1)×(M+1)(n+1) (25)
where C is a large positive constant. The important problem in this algorithm is theoretical convergence. Many researchers have reported on and analyzed [52], [53] the Kalman filter al-gorithm with a fixed input dimension. Moreover, the theoretical convergence analysis concerning varying input dimensions is an issue for another investigation. In order to experimentally address this problem, the matrix S must be reset to C· I. The aim of this resetting operation is to keep S bounded.
We now describe the antecedent parameter and compen-satory coefficient learning of the TSCIT2FNN based on the gradient descent algorithm as follows.
Let wi
j denote the antecedent parameters, m1, m2, and σ, in
the ith interval type-2 set in input variable xj wij(t + 1) = wji(t)− η ∂E ∂wi j(t) (26) where ∂E ∂wi j = (y− yd) " ∂y ∂ψi ∂ψi ∂fi ∂fi ∂μi j ∂μi j ∂wi j + ∂y ∂ψi ∂ψi ∂fi ∂fi ∂μij ∂μi j ∂wi j # (27) and η is a learning coefficient in the range [0,1].
∂y ∂ψi = q yi TSK−y M i=1 ψi ∂y ∂ψi = (1−q)y i TSK−y M i=1 ψi y= qy+(1−q)y= q M i=1 ψiyi TSK M i=1 ψi +(1−q) M i=1 ψiyi TSK M i=1 ψi . (28) In (28), yi
TSK represents the consequent value, which can be
seen in (12). If wi j = mij1, then we have ∂ψi ∂mi j1 =∂ψ i ∂fi ∂fi ∂μi j ∂μi j ∂mi j1 = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 1− γi+γi n (fi)−γi+γin ×(fi) xj−mij1 (σi j) 2 ! , xj> mi j1+mij2 2 0, otherwise (29) ∂ψi ∂mi j1 =∂ψ i ∂fi ∂fi ∂μi j ∂μij ∂mi j1 = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 1− γi+γi n (fi)−γi+γin ×(fi ) xj−m i j1 (σi j) 2 ! , xj ≤ mij1 0, otherwise. (30)
Similarly, if wji = mij2, then we have ∂ψi ∂mi j2 =∂ψ i ∂fi ∂fi ∂μi j ∂μi j ∂mi j2 = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1− γi+γi n ×(fi)−γi+γi n(fi) xj−m i j2 (σi j) 2 ! , xj ≤ mi j1+mij2 2 0, otherwise (31) ∂ψi ∂mi j2 =∂ψ i ∂fi ∂fi ∂μi j ∂μi j ∂mi j2 = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 1− γi+γi n (fi)−γi+γin ×(fi) xj−m i j2 (σi j) 2 ! , xj> mij1 0, otherwise. (32) If wi j= σij, then we have ∂ψi ∂σi j =∂ψ i ∂fi ∂fi ∂μi j ∂μij ∂σi j = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1− γi+γni (fi)−γi+γin ×(fi )(xj−m i j2) 2 (σi j) 3 , xj ≤ mi j1+m i j2 2 1− γi+γi n (fi)−γi+γin ×(fi)(xj−mij1) 2 (σi j) 3 , xj > mi j1+mij2 2 (33) ∂ψi ∂σi j =∂ψ i ∂fi ∂fi ∂μi j ∂μi j ∂σi j = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1− γi+γi n (fi)−γi+γin ×(fi )(xj−m i j1) 2 (σi j) 3 , xj < mij1 1− γi+γni (fi)−γi+γin ×(fi)(xj−m i j2) 2 (σi j) 3 , xj > mij2 0, otherwise. (34)
As mentioned earlier, the design factor q allows us to adjust the lower and upper portions in the final output according to the following formulas: q(t + 1) = q(t)− η∂E ∂q ∂E ∂q = (y − y d) ⎛ ⎜ ⎜ ⎜ ⎝ M i=1 ψiyi TSK M i=1 ψi − M i=1 ψiyi TSK M i−1ψ i ⎞ ⎟ ⎟ ⎟ ⎠. (35)
Next, we describe the relative compensatory coefficient γithat
is updated by the gradient descent algorithm ∂E ∂γi = ∂E ∂y " ∂y ∂ψi ∂ψi ∂γi + ∂y ∂ψi ∂ψi ∂γi # =∂E ∂y " ∂y ∂ψi ln|fi|(ψi) 1 n− 1 ! + ∂y ∂ψi ln|fi| (ψi) 1 n− 1 !# . (36) Then, the parameters ci and di are the relative coefficients of
the compensatory degree γi ci(t + 1) = ci(t) + ηc + 2ci(t)d2i(t) [c2 i(t) + (d2i(t)] 2 , ∂E ∂γi (37) di(t + 1) = di(t)− ηd + 2c2 i(t)di(t) [c2 i(t) + (d2i(t)] 2 , ∂E ∂γi (38) γi(t + 1) = c 2 i(t + 1) c2 i(t + 1) + d2i(t + 1) (39) where the different learning rates, i.e., ηcand ηd, enable us to
adapt the compensatory degree more efficiently.
The gradient descent algorithm is used in a TSCIT2FNN for online learning, although it may have defective characteristics in terms of learning convergence and the local minimum prob-lem. Convergence is guaranteed by the following condition:
0 < η(t) < 2 max t ---∂y(t) ∂W ---.
The derivation of this condition can be found in [48] and [65] and is given in the Appendix.
The learning rate η is within the range [0, 1]. A large value results in extreme oscillations of the updated parameters. On the other hand, a small learning rate makes the learning convergence slower. It is important that the proper learning rate be chosen to adapt the system.
IV. SIMULATIONS
This section describes five examples to evaluate the perfor-mance of the TSCIT2FNN. These simulation studies include three types of system identification, ANC, and prediction of the Mackey–Glass time series. As a result, the performance of our TSCIT2FNN is compared with that of other existing type-1 and type-2 FNNs.
A. Example 1 (System Identification)
The TSCIT2FNN is applied to the identification of a non-linear system, which has been presented in [3]. The process is described by the difference equation
yd(t + 1) = yd(t)
1 + y2 d(t)
+ u3(t). (40)
The training signal is generated with u(t) = sin(2πt/100), t = 1, . . . , 200. The TSCIT2FNN inputs are u(t) and yd(t), and
Fig. 3. Simulation curve of the relative weights of compensatory operation during learning.
Fig. 4. Identification result of the TSCIT2FNN. TABLE I
PERFORMANCECOMPARISON FOREXAMPLE1
the desired output is yd(t + 1). As a performance criterion, we
evaluate the root-mean-square error (rmse)
rmse = . / / 0 1 200 200 t=1 [y(t + 1)− yd(t + 1)] (41)
where y(t + 1) is the TSCIT2FNN output. The learning rate η is set to 0.075. The structure threshold fth determines the
number of rules. When fth is set to 0.02, three fuzzy rules
are constructed. Training is performed for 500 epochs with 200 steps. After training, the rmse obtained is 0.0078. All networks use the identical training, test data, and number of training epochs as the TSCIT2FNN. Fig. 3 shows the relative coef-ficients of a compensatory operation during training epochs. Fig. 4 shows the TSCIT2FNN results for the test.
This example also computes the performance of TSCIT2FNN with different models as shown in Table I. Table I shows the network performance, including the network size, training rmse, and test rmse, and compares them with other type-1 and type-2 FNNs presented in [3], [44], and [48].
The feedforward type-1 FNN that we use for comparison is a SONFIN [3], a powerful network with both structure and parameter learning abilities. The consequent part of the SON-FIN is of the Mamdani type. All free parameters in SONSON-FIN are learned using the gradient descent algorithm. The original interval type-2 FNN for comparison is presented in [39], where the type-2 FLS structure is fixed and assigned in advance. In an attempt to have similar network size for comparison, the total number of parameters in a feedforward type-1 FNN is designed to be close to that in a feedforward type-2 FNN.
The type-2 TSK FNS [48] has fewer parameters than other type-2 FNNs because the type-2 TSK FNS uses a crisp linear function in the consequent part, as does the TSCIT2FNN. The feedforward type-2 FNN, being of the TSK type in the consequent part, is similar to the SEIT2FNN structure because they use simultaneous structure and parameter learning online. However, the feedforward type-2 FNN uses a gradient descent algorithm to tune the free parameters. As can be seen in Table I, the performance of the feedforward type-2 FNN is inferior to that of the SEIT2FNN using gradient descent and Kalman filter algorithms. The type-2 TSK FLS must define rules in advance and uses the FCM to construct rules. For parameter learning, the type-2 TSK FLS and type-2 TSK FNS use the gradient descent algorithm to achieve the optimal values. In comparison with the type-2 models, the SEIT2FNN achieves a smaller test rmse than the other models. This is because the SEIT2FNN has distinguishing characteristics for obtaining a better performance, including a self-evolving ability and the use of a rule-ordered Kalman filter algorithm to tune interval weights in the consequent part.
In general, the type-reduction process in the feedforward type-2 models, i.e., the type-2 TSK FLS and the SEIT2FNN [44], is more difficult and time consuming. We now analyze the practical computational cost of constructing a TSCIT2FNN. All simulations are performed on an Intel 2.53-GHz dual central processing unit, and the programs are written in Visual C++. The execution time for the learning process of the TSCIT2FNN is 1.14 s. The learning times of three representative models, SONFIN, type-2 TSK FLS, and SEIT2FNN, take 0.7, 1.55, and 1.42 s, respectively. The results show that the computational cost of our TSCIT2FNN is less than that of two type-2 models. We now compare the performance of SEIT2FNN and type-2 TSK FNS with our approach. The test error of TSCIT2FNN is close to that of SEIT2FNN but has fewer parameters. In contrast to the type-2 TSK FNS [48], our network yields a lower test rmse, as can be seen in Table I. Both TSCIT2FNN and type-2 TSK FNS obtain their final output based on design factor q. The results show that the TSCIT2FNN rmse is smaller than that of the other type-2 models, except the SEIT2FNN.
For the further simulation studies, the modification plant is described by yd(t + 1) = α(t)yd(t) 1 + y2 d(t) + u3(t) (42) where α(t) = 1 1, 1≤ t ≤ 1000 sin(t/100), t > 1000. (43)
Fig. 5. RMSE values obtained from online learning.
Fig. 6. Online identification results of the actual output and the TSCIT2FNN. The signal u(t) = sin(2πt/100) is applied to the system in (42) as input, and then, three rules are generated after 1000 time steps of online learning. The time-varying system continuously changes after 1000 time steps, and additional five rules are generated and cover the following new inputs. Fig. 5 shows the TSCIT2FNN rmses that are accumulated over 100 adjacent time steps. Fig. 6 shows the learning result of the actual output and the TSCIT2FNN. The results demonstrate the TSCIT2FNN can efficiently handle time-varying system with better performance.
B. Example 2 (Second-Order System Identification)
This example uses the TSCIT2FNN to identify a second-order time-varying system, a problem that was introduced in [48] and [64]. The dynamic system is described by the follow-ing difference equation:
yp(t + 1) = f (yp(t), yp(t−1), yp(t−2), u(t), u(t−1)) (44) where f (x1, x2, x3, x4, x5) = x1x2x3x5(x3− b) + cx4 a + x2 2+ x23 . (45) The time-varying parameters a(t), b(t), and c(t) are given by
a(t) = 1.2− 0.2 cos(2πt/T ) (46) b(t) = 1.0− 0.4 sin(2πt/T ) (47) c(t) = 1.0 + 0.4 sin(2πt/T ) (48) where T is the maximum time step.
Fig. 7. Identification result of the TSCIT2FNN using three rules.
TABLE II
PERFORMANCECOMPARISON FOREXAMPLE2
The dynamic system output depends on three previous out-puts and one previous input. In this example, only two current values, u(t) and yp(t), are fed as input to the TSCIT2FNN
input layer. In the training procedure of the TSCIT2FNN, we follow the same computational protocols as in [48]. After 100 epochs of training, three rules are generated when the structure-learning threshold is set to 0.04. The structure-learning coefficient is set to 0.07. To evaluate the identification result, the following signal is used for the test:
u(t) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ sin πt25, t < 250 1.0, 250≤t<500 −1.0, 500≤t<750 0.3 sin πt 25 +0.1 sin πt 32 +0.6 sin πt10, 750≤t<1000. (49)
Fig. 7 shows the outputs of the plant and the TSCIT2FNN for these test inputs.
The three models, type-1 and type-2 TSK FNSs and the SEIT2FNN, use an identical number of input variables, training data, test data, and training epochs as the TSCIT2FNN. For comparison with test rmse, they are almost identical. Neither the type-2 TSK FNS nor the SEIT2FNN use the compen-satory operation. The parameter learning of the type-1 TSK FNS is trained using gradient descent-based optimization. Al-though the feedforward models were unable to effectively handle dynamic systems, the results in Table II indicate that the TSCIT2FNN achieves similar performance with fewer rules.
Fig. 8. Simulation curve of the TSCIT2FNN output.
Fig. 9. Simulation curve of absolute relative error between actual output and the TSCIT2FNN.
TABLE III
PERFORMANCECOMPARISON FOREXAMPLE3
C. Example 3 (Nonlinear Function Identification)
To assess the validity of the TSCIT2FNN, a data synthesis is made with the following function:
y =264− 81 · ((x1− 0.6)2+ (x2− 0.5)2/9− 0.5. (50)
This plant has two inputs (x1 and x2) and one output y, so
the two input variables are fed as inputs to the TSCIT2FNN input layer. As in [54], the inputs are randomly generated within [0, 1]. To generate 1100 patterns, the first 1000 patterns are used for training, and the remaining 100 patterns are used for testing. The structure threshold fth, learning rate η, and design
factor q are set to 0.05, 0.075, and 0.3, respectively. After 100 epochs of training, three rules are generated. Fig. 8 depicts the actual output and the TSCIT2FNN output response for the test patterns. The absolute relative error of the TSCIT2FNN is also shown in Fig. 9. Table III shows the maximum, minimum, and average relative error. The average relative error of the
TSCIT2FNN is less than 2%. We also compare the performance of the SEIT2FNN [44] and TSCIT2FNN. As can be seen from Table III, the average relative error of the TSCIT2FNN is similar to that of the SEIT2FNN. Indeed, the results in Table III reveal the advantage of the TSCIT2FNN. As mentioned ear-lier, the combination of technologies in the TSCIT2FNN, like compensatory operation and the use of a design factor to adjust portions of the lower and upper, provides more accurate results, which is not the case with SEIT2FNN.
Table III compares the performance of the TSCIT2FNN with that of existing type-1 FNNs, including an Elman-style recurrent network (ENN) [55], a fuzzy-rule radial basis function neural network [54], a recurrent FNN (RFNN) [56], and a wavelet FNN (WFNN) [12]. These models used an identical number of training epochs, rules, and training and test samples as the TSCIT2FNN. Of the type-1 FNNs, the WFNN achieves the smallest relative error. This is because the consequent part of the WFNN consists of wavelet basis functions, which have the ability to localize both in the time and frequency domains. The results indicate that the TSCIT2FNN obtains better perfor-mance than the other networks.
D. Example 4 (ANC)
In this example, we consider noise tolerance and system recovery. The TSCIT2FNN is applied to the ANC problem presented in [57] and [58]. ANC is concerned with the enhance-ment of noise corrupted signals and has been used successfully in various areas, such as interference cancellation in elec-trocardiograms, echo elimination on long-distance telephone transmission lines, and antenna interference canceling.
ANC is based on the availability of a primary input source and an auxiliary (reference) input source located at the noise field that contains little or none, as shown in Fig. 10. In this figure, the primary input source contains the desired output s and the noise n0generated from the noise source n in order to
corrupt the desired output. The received signal is thus
x(t) = s(t) + n0(t). (51)
The secondary or auxiliary (reference) input source receives the noise n1, which is correlated with the corrupting noise n0.
The purpose of ANC techniques is to adaptively process the reference noise n0, which is generated as a replica of noise n1,
and then subtract this from the primary input x to recover the desired signal s. The TSCIT2FNN output y represents a replica of n0. In [58], the assumptions are made that s, n0, and n1
are stationary zero-mean processes, s is independent, and n0
and n1are dependent. Therefore, the reference input source is
situated in such a position that it detects only the noise but not the signal s. From Fig. 10, we can suppose that
e(t) = s(t) + n0(t)− y(t) (52)
where
s(t) = 0.6 sin(0.06t) cos(0.01t) (53)
Fig. 10. ANC using the TSCIT2FNN.
Fig. 11. (a) Original signal source s. (b) Corrupted signal x. (c) Recovered signal e.
and the noise source n is white noise with a uniform distribution in [−1.5, 1.5]. The relation between the noise source n and the corrupting noise n1is a nonlinear function that is expressed as
n0= 0.6 (n(t))3. (54)
The reference input is placed in front of the noise source so that n = n1. The reference input n1 is fed as input, and then,
the actual output of the TSCIT2FNN is derived. The learning coefficient η is set to 0.01. The TSCIT2FNN generates seven rules during the online training process when the structure-learning threshold fth is set to 0.5. Fig. 11 shows the
de-tailed training process from time step 48 000 to 50 000, where Fig. 11(a)–(c) shows the original, corrupted, and recovered signals, respectively. The performance of the TSCIT2FNN is compared to that of the adaptive neural fuzzy filter (AFNN) [57] and the SEIT2FNN [44]. The AFNN is a type-1 fuzzy structure with both structure and parameter learning abilities. Both AFNN and SEIT2FNN generate seven rules during the online training process.
The square errors between the recovered and original sig-nals are computed such that each plotted error value is the sum of square errors over 60 adjacent time steps for each network, as shown in Fig. 12. This figure illustrates that the TSCIT2FNN has faster convergence than the other models. The result shows that the two type-2 structures, i.e., SEIT2FNN and TSCIT2FNN, are better than the type-1 structure (AFNN) with identical fuzzy rules. Fig. 12 illustrates that the TSK-type models with interval weights in the consequent part are susceptible to noisy environments.
Fig. 12. Square error compared with the different models.
Fig. 13. Prediction results using the TSCIT2FNN.
E. Example 5 (Prediction of a Chaotic Time Series)
This example considers a prediction problem that has been used in [44] and [59]–[63]. The proposed TSCIT2FNN model is applied to the Mackey–Glass time series that is generated from the following delay differential equation:
dx(t)
dt =
0.2x(t− τ)
1 + x10(t− τ)− 0.1x(t). (55)
For τ = 17, the system response is chaotic, and simulation data are obtained using the initial conditions x(0) = 1.2 and τ = 17, as in the previous literature. We generate 1000 input–output data points from t = 124 to 1123, with the first 500 patterns being used for training and the remaining 500 for testing. Four past values are used to predict x(t), and the input–output data format is [x(t− 24), x(t − 18), x(t − 12), x(t − 6); x(t)].
The structure-learning threshold fth, learning factor η, and
design factor q are set to 0.2, 0.075, and 0.3, respectively. After 500 training epochs, seven rules are generated. Fig. 13 shows the prediction result of the TSCIT2FNN. Fig. 14 depicts the prediction error between the actual result and that of the TSCIT2FNN, which shows that there is an excellent match between them.
A comparison of the TSCIT2FNN model with different models, including the radial based adaptive fuzzy system (RBF-AFS) [59], hybrid neural fuzzy inference system (HyFIS) [60], a neuro-fuzzy function approximator (NEFPROX) [61], dy-namic FNN (DFNN) [62], FWNN [11], local linear wavelet neural network with hybrid model (LLWNN+hybrid) [63], and SEIT2FNN [44], is illustrated in Table IV. For a fair com-parison, the proposed approach is compared with SEIT2FNN, which also uses a type-2 fuzzy set with uncertain means in
Fig. 14. Test error of the chaotic time-series prediction using the TSCIT2FNN.
TABLE IV
PERFORMANCECOMPARISON FOREXAMPLE5
the antecedent part. The result indicates that the SEIT2FNN performs better than the proposed approach. The reason for this is that the SEIT2FNN achieves better performance in the consequent parts using interval type-1 sets. However, the proposed approach has fewer parameters than the SEIT2FNN and is thus less computationally expensive.
The performance of the TSCIT2FNN is also comparable to that of recently developed type-1 FNNs. As can be seen from Table IV, the FWNN is one of the best models in terms of low test rmse. For their consequent parts, both the FWNN and the LLWNN are composed of wavelet basis functions, which have the ability to localize both in the time and frequency domains. A hybrid algorithm of particle swarm optimization and gradient descent is used for training the LLWNN. A fast gradient-based training algorithm, the Broyden–Fletcher–Goldfarb–Shanno method, is used to find optimal values for free parameters of the FWNN models. However, the performance of the TSCIT2FNN is close to the FWNN’s performance, due in part to having fewer rules. Finally, the results indicate that the TSCIT2FNN achieves a smaller prediction error and has fewer rules than type-1 fuzzy models.
We now analyze the practical computational cost of con-structing a TSCIT2FNN. The execution time for the learning process of the TSCIT2FNN is 9.26 s. The SEIT2FNN takes 37.3 s. The results show that the computational cost of the TSCIT2FNN is much less than that of the SEIT2FNN.
V. CONCLUSION
In this paper, a TSK-type-based compensatory interval type-2 FNN with a self-evolving structure and parameter learning was proposed. The FoU of type-2 fuzzy sets in the
TSCIT2FNN enables the system to handle the numerical un-certainty as well as uncertain information. To obtain the optimal type-2 fuzzy logic reasoning and select the optimal type-2 fuzzy operation, we have exploited a compensatory operator. Thus, our TSCIT2FNN can adaptively tune the type-2 fuzzy MF and efficiently optimize the fuzzy operator. In terms of structure learning, there is no need to decide the TSCIT2FNN rules in advance. The TSCIT2FNN provides the self-evolving ability to structure itself online, which is especially useful for handling industrial problems with temporally varying characteristics. The premise and consequent parameters are learned as gradient descent and variable-expansive Kalman filter algorithms to improve the learning accuracy. Finally, the TSCIT2FNN has potential to deal with noisy data, as verified in the simulation results. In summary, the TSCIT2FNN variance system achieved a better performance than existing type-1 or type-2 FNNs in identification, prediction, and ANC.
APPENDIX
This appendix derives the optimal learning rate using a Lyapunov function by following the previous studies [48], [65]. Let η(t) be the learning rate at discrete time t for the update formula of the TSCIT2FNN weights. The convergence is guaranteed if η(t) is satisfied by the following condition:
0 < η(t) < 2 max t ---∂y(t) ∂W ---. (A1)
The statement in (A1) can be proved by choosing the following Lyapunov function:
V (t) = 1 2e
2(t), e(t) = (y
d(t)− y(t)) . (A2)
The change in the Lyapunov function is obtained as ΔV (t) = V (t + 1)− V (t) = 1 2 e2(t + 1)− e2(t) =1 2 (e(t) + Δe(t))2− e2(t) =1 2
2e(t)· Δe(t) + Δe2(t)
=1
2Δe(t) (2e(t) + Δe(t)) . (A3)
The error difference can be represented as Δe(t) = ∂e(t) ∂W !T ΔW = 3 ∂ (yd(t)− y(t)) ∂W 4T ΔW = − 3 ∂y(t) ∂W 4 ΔW. (A4)
From the update formulas, we get
ΔW =−η∂E ∂W = ηe(t) ∂y(t) ∂W where ∂E ∂W = ∂ ∂W 3 1 2e 2(t) 4 = e(t)∂e(t) ∂W =−e(t) ∂y(t) ∂W (A5) ΔV (t) =1
2Δe(t) (2e(t) + Δe(t)) =1 2η(t)e 2(t)----∂y(t) ∂W ----2 η(t)----∂y (t) ∂W ----2− 2 . (A6)
From the Lyapunov stability theorem, asymptotic stability is guaranteed under the following sufficient condition:
0 < η(t) < 2 max t ---∂y(t) ∂W ---. (A7) REFERENCES
[1] L. A. Zadeh, “The concept of linguistic variable and its application to approximate reasoning,” Inf. Sci., vol. 8, no. 3, pp. 199–249, 1975. [2] J. S. R. Jang, C. T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft
Comput-ing. Englewood Cliffs, NJ, USA: Prentice-Hall, 1997, ch. 17.
[3] C. F. Juang and C. T. Lin, “An on-line self-constructing neural fuzzy inference network and its applications,” IEEE Trans. Fuzzy Syst., vol. 6, no. 1, pp. 12–32, Feb. 1998.
[4] R. J. Wai and P. C. Chen, “Intelligent tracking control for robot manip-ulator including actuator dynamics via TSK-type fuzzy neural network,”
IEEE Trans. Fuzzy Syst., vol. 12, no. 4, pp. 552–560, Aug. 2004.
[5] Y. X. Liao, J. H. She, and M. Wu, “Integrated hybrid-PSO and fuzzy-NN decoupling control for temperature of reheating furnace,” IEEE Trans.
Ind. Electron., vol. 56, no. 7, pp. 2704–2714, Jul. 2009.
[6] C. T. Lin, C. M. Yeh, S. F. Liang, J. F. Chung, and N. Kumar, “Support-vector-based fuzzy neural network for pattern classification,” IEEE Trans.
Fuzzy Syst., vol. 14, no. 1, pp. 31–41, Feb. 2006.
[7] L. Y. Shyu, Y. H. Wu, and W. Hu, “Using wavelet transform and fuzzy neural network for VPC detection from the holter ECG,” IEEE Trans.
Biomed. Eng., vol. 51, no. 7, pp. 1269–1273, Jul. 2004.
[8] I. S. Li, W. Y. Wang, S. F. Su, and Y. S. Lee, “A merged fuzzy neural network and its applications in battery state-of-charge estimation,” IEEE
Trans. Energy Convers., vol. 22, no. 3, pp. 697–708, Sep. 2007.
[9] A. Rubaai, A. R. Ofoli, L. Burge, III, and M. Garuba, “Hardware im-plementation of an adaptive network-based fuzzy controller for dc-dc converters,” IEEE Trans. Ind. Appl., vol. 41, no. 6, pp. 1557–1565, Nov./Dec. 2005.
[10] R. H. Abiyev and O. Kaynak, “Fuzzy wavelet neural networks for identi-fication and control of dynamic plants—A novel structure and a compar-ative study,” IEEE Trans. Ind. Electron., vol. 55, no. 8, pp. 3133–3140, Aug. 2008.
[11] S. Yilmaz and Y. Oysal, “Fuzzy wavelet neural network models for predic-tion and identificapredic-tion of dynamical systems,” IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1599–1609, Oct. 2010.
[12] C. H. Lu, “Wavelet fuzzy neural network for identification and predictive control of dynamic systems,” IEEE Trans. Ind. Electron., vol. 58, no. 7, pp. 3046–3058, Jul. 2011.
[13] H. J. Zimmermann and P. Zysno, “Latent connective in human decision making,” Fuzzy Sets Syst., vol. 4, no. 1, pp. 37–51, Jul. 1980.
[14] Y. Q. Zhang and A. Kandel, “Compensatory neurofuzzy systems with fast learning algorithm,” IEEE Trans. Neural Netw., vol. 9, no. 1, pp. 83–105, Jan. 1998.
[15] S. Halgamuge and M. Glesner, “Neural networks in designing fuzzy sys-tems for real world application,” Fuzzy Sets Syst., vol. 65, no. 1, pp. 1–12, Jul. 1994.
[16] N. Kasabov, “Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid system,” Fuzzy Sets Syst., vol. 82, no. 2, pp. 135–149, Sep. 1996.
[17] D. Nauck and R. Kruse, “A neuro-fuzzy method to learn fuzzy classi-fication rules from date,” Fuzzy Sets Syst., vol. 89, no. 3, pp. 277–288, Aug. 1997.
[18] S. Paul and S. Kumar, “Subsethood-product fuzzy neural inference system (SuPFuNIS),” IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 578–599, May 2002.
[19] J. S. Wang and G. C. S. Lee, “Self-adaptive neuro-fuzzy inference system for classification applications,” IEEE Trans. Fuzzy Syst., vol. 10, no. 6, pp. 790–802, Dec. 2002.
[20] H. Seker, D. E. Evans, N. Aydin, and E. Yazgan, “Compensatory fuzzy neural networks-based intelligent detection of abnormal neonatal cere-bral Doppler ultrasound waveforms,” IEEE Trans. Inf. Technol. Biomed., vol. 5, no. 3, pp. 187–194, Sep. 2001.
[21] C. J. Lin and W. H. Ho, “A pseudo-Gaussian-based compensatory neural fuzzy system,” in Proc. IEEE Int. Conf. Fuzzy Syst., May 25–28, 2003, pp. 214–219.
[22] C. J. Lin and C. H. Chen, “Nonlinear system control using com-pensatory neuro-fuzzy networks,” IEICE Trans. Fundam. Electron.
Commun. Comput. Sci., vol. E86-A, no. 9, pp. 2309–2316, 2003.
[23] C. H. Chen, C. J. Lin, and C. T. Lin, “An efficient quantum neuro-fuzzy classifier based on neuro-fuzzy entropy and compensatory operation,” Soft
Comput., vol. 12, no. 6, pp. 567–583, Jan. 2008.
[24] C. S. Ouyang and S. J. Lee, “An improved learning algorithm for rule refinement in neuro-fuzzy modeling,” in Proc. 3rd Int. Conf.
Knowl.-based Intell. Inf. Eng. Syst., 1999, pp. 238–241.
[25] N. N. Karnik, J. M. Mendel, and Q. Liang, “Type-2 fuzzy logic systems,”
IEEE Trans. Fuzzy Syst., vol. 7, no. 6, pp. 643–658, Dec. 1999.
[26] J. M. Mendel and R. I. John, “Type-2 fuzzy sets made simple,” IEEE
Trans. Fuzzy Syst., vol. 10, no. 2, pp. 117–127, Apr. 2002.
[27] C. Wagner and H. Hagras, “Toward general type-2 fuzzy logic systems based on zSlices,” IEEE Trans. Fuzzy Syst., vol. 18, no. 4, pp. 637–660, Aug. 2010.
[28] D. Wu and J. M. Mendel, “Computing with words for hierarchical deci-sion making applied to evaluating a weapon system,” IEEE Trans. Fuzzy
Syst., vol. 18, no. 3, pp. 441–460, Jun. 2010.
[29] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic System:
Introduc-tion and New DirecIntroduc-tions. Upper Saddle River, NJ, USA: Prentice-Hall,
2001.
[30] X. X. Zhang, H. X. Li, and C. K. Qi, “Spatially constrained fuzzy-clustering-based sensor placement for spatiotemporal fuzzy-control system,” IEEE Trans. Fuzzy Syst., vol. 18, no. 5, pp. 946–957, Oct. 2010.
[31] J. M. Mendel, “Comments on ‘A-plane representation for type-2 fuzzy sets: Theory and applications’,” IEEE Trans. Fuzzy Syst., vol. 18, no. 1, pp. 229–230, Feb. 2010.
[32] A. Niewiadomski, “On finity, countability, cardinalities, and cylindric extensions of type-2 fuzzy sets in linguistic summarization of databases,”
IEEE Trans. Fuzzy Syst., vol. 18, no. 3, pp. 532–545, Jun. 2010.
[33] J. Aisbett, J. T. Rickard, and D. G. Morgenthaler, “Type-2 fuzzy sets as functions on spaces,” IEEE Trans. Fuzzy Syst., vol. 18, no. 4, pp. 841–844, Aug. 2010.
[34] H. Hagras, “A hierarchical type-2 fuzzy logic control architecture for autonomous mobile robots,” IEEE Trans. Fuzzy Syst., vol. 12, no. 4, pp. 524–539, Aug. 2004.
[35] E. Kayacan, O. Cigdem, and O. Kaynak, “Sliding mode control approach for online learning as applied to type-2 fuzzy neural networks and its experimental evaluation,” IEEE Trans. Ind. Electron., vol. 59, no. 9, pp. 3510–3520, Sep. 2012.
[36] C. F. Juang and C. H. Hsu, “Reinforcement ant optimized fuzzy controller for mobile-robot wall-following control,” IEEE Trans. Ind. Electron., vol. 56, no. 10, pp. 3931–3940, Oct. 2009.
[37] C. S. Lee, M. H. Wang, and H. Hagras, “A type-2 fuzzy ontology and its application to personal diabetic-diet recommendation,” IEEE Trans.
Fuzzy Syst., vol. 18, no. 2, pp. 374–395, Apr. 2010.
[38] M. A. Khanesar, E. Kayacan, M. Teshnehlab, and O. Kaynak, “Analysis of the noise reduction property of type-2 fuzzy logic system using a novel type-2 membership function,” IEEE Trans. Syst., Man, Cybern. B,
Cybern., vol. 41, no. 5, pp. 1395–1406, Oct. 2011.
[39] C. H. Wang, C. S. Cheng, and T. T. Lee, “Dynamical optimal training for interval type-2 fuzzy neural network,” IEEE Trans. Syst., Man, Cybern.
B, Cybern., vol. 34, no. 3, pp. 1462–1477, Jun. 2004.
[40] Y. C. Lin and C. H. Lee, “Type-2 fuzzy neural network systems and learning,” Int. J. Comput. Cognit., vol. 1, no. 4, pp. 79–90, Dec. 2003.
[41] H. Hagras, “Comments on dynamical optimal training for interval type-2 fuzzy neural network (Ttype-2FNN),” IEEE Trans. Syst., Man, Cybern. B,
Cybern., vol. 36, no. 5, pp. 1206–1209, Oct. 2006.
[42] J. M. Mendel, “Computing derivatives in interval type-2 fuzzy logic sys-tems,” IEEE Trans. Fuzzy Syst., vol. 12, no. 1, pp. 84–98, Feb. 2004. [43] C. Y. Yeh, W. R. Jeng, and S. J. Lee, “Data-based system
model-ing usmodel-ing a type-2 fuzzy neural network with a hybrid learnmodel-ing al-gorithm,” IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 2296–2309, Dec. 2011.
[44] C. F. Juang and Y. W. Tsao, “A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning,” IEEE Trans. Fuzzy
Syst., vol. 16, no. 6, pp. 1411–1424, Dec. 2008.
[45] J. Y. Chang, Y. Y. Lin, M. F. Han, and C. T. Lin, “A functional-link based interval type-2 compensatory fuzzy neural network for nonlinear system modeling,” in Proc. Int. Conf. Fuzzy Syst., Jun. 27–30, 2011, pp. 939–943.
[46] C. F. Juang, R. B. Huang, and Y. Y. Lin, “A recurrent self-evolving interval type-2 fuzzy neural network for dynamic system processing,” IEEE Trans.
Fuzzy Syst., vol. 17, no. 5, pp. 1092–1105, Oct. 2009.
[47] J. R. Castro, O. Castillo, P. Melin, and A. Rodrigue-Diaz, “A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks,”
Inf. Sci., vol. 179, no. 13, pp. 2175–2193, Jun. 2009.
[48] R. H. Abiyev and O. Kaynak, “Type 2 fuzzy neural structure for identi-fication and control of time-varying plants,” IEEE Trans. Ind. Electron., vol. 57, no. 12, pp. 4147–4159, Dec. 2010.
[49] F. J. Lin and P. H. Chou, “Adaptive control of two-axis motion control system using interval type-2 fuzzy neural network,” IEEE Trans. Ind.
Electron., vol. 56, no. 1, pp. 178–193, Jan. 2009.
[50] O. Uncu and I. B. Truksen, “Discrete interval type 2 fuzzy system mod-els using uncertainty in learning parameters,” IEEE Trans. Fuzzy Syst., vol. 15, no. 1, pp. 90–106, Feb. 2007.
[51] M. A. Khanesar, E. Kayacan, M. Teshnehiab, and O. Kaynak, “Extended Kalman filter based learning algorithm for type-2 fuzzy logic systems and its experimental evaluation,” IEEE Trans. Ind. Electron., vol. 59, no. 11, pp. 4443–4455, Nov. 2012.
[52] L. Guo, “Estimating time-varying parameters by the Kalman filter based algorithm: Stability and convergence,” IEEE Trans. Autom. Control, vol. 35, no. 2, pp. 141–147, Feb. 1990.
[53] J. J. Rubio and W. Yu, “Nonlinear system identification with recurrent neural networks and dead-zone Kalman filter algorithm,”
Neurocomput-ing, vol. 70, no. 13–15, pp. 2460–2466, Aug. 2007.
[54] W. Li and Y. Hori, “An algorithm for extracting fuzzy rules based on RBF neural network,” IEEE Trans. Ind. Electron., vol. 53, no. 4, pp. 1269– 1276, Jun. 2006.
[55] S. C. Kremer, “On the computational power of Elman-style recurrent networks,” IEEE Trans. Neural Netw., vol. 6, no. 4, pp. 1000–1004, Jul. 1995.
[56] C. H. Lu and C. C. Tsai, “Generalized predictive control using recur-rent fuzzy neural networks for industrial processes,” J. Process Control, vol. 17, no. 1, pp. 83–92, Jan. 2007.
[57] C. T. Lin and C. F. Juang, “An adaptive neural fuzzy filter and its ap-plications,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 27, no. 4, pp. 635–656, Aug. 1997.
[58] O. Castillo and P. Melin, “Adaptive noise cancellation using type-2 fuzzy logic and neural networks,” in Proc. IEEE Int. Conf. Fuzzy Syst., Jul. 2004, vol. 2, pp. 1093–1098.
[59] K. B. Cho and B. H. Wang, “Radial basis function based adaptive fuzzy systems and their applications to system identification,” Fuzzy Sets Syst., vol. 83, no. 3, pp. 325–339, Nov. 1996.
[60] J. Kim and N. K. Kasabov, “HyFIS: Adaptive neuro-fuzzy inference sys-tems and their application to nonlinear dynamic syssys-tems,” Neural Netw., vol. 12, no. 9, pp. 1301–1319, Nov. 1999.
[61] D. Nauk and R. Kruse, “Neuro-fuzzy systems for function approxima-tion,” Fuzzy Sets Syst., vol. 101, no. 2, pp. 261–271, Jan. 1999. [62] S. Wu and M. J. Er, “Dynamic fuzzy neural networks—A novel approach
to function approximation,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 30, no. 2, pp. 358–364, Apr. 2000.
[63] Y. Chen, B. Yang, and J. Dong, “Time-series prediction using a local linear wavelet neural network,” Neurocomputing, vol. 69, no. 4–6, pp. 449–465, Jan. 2006.
[64] Y. Takahashi, “Adaptive predictive control of nonlinear time varying sys-tems using neural networks,” in Proc. IEEE Int. Conf. Neural Netw., 1993, pp. 1464–1468.
[65] Y. C. Lin and C. H. Lee, “System identification and adaptive filter using a novel fuzzy neuro system,” Int. J. Comput. Cognit., vol. 5, no. 1, pp. 15– 26, Mar. 2007.
Yang-Yin Lin received the B.S. degree from
the Department of Electronics Engineering, Na-tional Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan, in 2005 and the M.S. degree from the Institute of Electrical Engineering, National Chung Hsing University, Taichung, Taiwan, in 2008. He is currently working toward the Ph.D. degree in the Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan.
His current research interests include computa-tional intelligence, type-2 fuzzy neural networks, and field-programmable gate array chip design.
Jyh-Yeong Chang (S’84–M’86) received the B.S.
degree in control engineering and the M.S. degree in electronic engineering from National Chiao Tung University, Hsinchu, Taiwan, in 1976 and 1980, re-spectively, and the Ph.D. degree in electrical en-gineering from North Carolina State University, Raleigh, NC, USA, in 1987.
During 1976–1978 and 1980–1982, he was a Re-search Fellow at the Chung Shan Institute of Science and Technology, Lung-Tan, Taiwan. Since 1987, he has been an Associate Professor with the Department of Electrical Engineering, National Chiao Tung University, where he is cur-rently a Professor. His current research interests include neural fuzzy systems, video processing and surveillance, and e-health systems.
Chin-Teng Lin (S’88–M’91–SM’99–F’05) received
the B.S. degree from National Chiao Tung University (NCTU), Hsinchu, Taiwan, in 1986 and the M.S. and Ph.D. degrees in electrical engineering from Purdue University, West Lafayette, IN, USA, in 1989 and 1992, respectively.
He is currently the Provost, Chair Professor of Electrical and Computer Engineering, and Direc-tor of the Brain Research Center, NCTU. He has published more than 120 journal papers in the ar-eas of neural networks, fuzzy systems, multimedia hardware/software, and cognitive neuro-engineering, including approximately 74 IEEE journal papers. He is the coauthor of Neural Fuzzy Systems (Prentice-Hall, 1996) and the author of Neural Fuzzy Control Systems with Structure and
Parameter Learning (World Scientific, 1994).
Dr. Lin serves as the Editor-in-Chief of the IEEE TRANSACTIONS ON
FUZZY SYSTEMS. He also served on the Board of Governors of the IEEE Circuits and Systems Society in 2005–2008, IEEE Systems, Man, and Cy-bernetics Society in 2003–2005, and IEEE Computational Intelligence Society in 2008–2010 and was the Chair of the IEEE Taipei Section in 2009–2010. He served as the Deputy Editor-in-Chief of the IEEE TRANSACTIONS ON
