1. Introduction
1.3 Dissertation Overview
The rest of this dissertation is organized as follows. Chapter 2 describes the design procedure of an adaptive self-structuring asymmetric fuzzy neural-network control for the static neural network. The training algorithms of parameters, including means and variances of membership functions and weights of the NN, are developed. The stability analysis and example illustrations are also provided in this chapter. For the DNN, the Hopfield-based DNN identifier is developed in Chapter 3. The training algorithm of weighting factors of the DNN is investigated. The stability analysis and example illustrations are also provided in this
chapter. The software and hardware of the implementation comparison between SFNN and Hopfield DNN is provided in Chapter 4. Finally, conclusions with future works are included in Chapter 5.
Chapter 2
Adaptive Self-structuring Asymmetric Fuzzy Neural-network Control Design
According to the used types of neural networks (NNs), they can be qualified as static (feed-forward) or as dynamic (recurrent) nets. In this chapter, the development of the static NN is priority to be discussed. The control design of fuzzy neural network (FNN) is explored first. The stability of the control system and examples will be also illustrated in this chapter.
2.1 Problem Statement
Consider the nth-order nonlinear dynamic system of the form u
f
x(n) = (x)+ (2-1) where x=[x x&Lx(n−1)]T, which is assumed to be available for measurement, is the state vector of the system, f(x) is the system dynamics equation, and u is the control effort.
The control objective is to find a control law so that the state trajectory x can track a command trajectory xc, and thus a tracking error is defined as
x x
e= c− . (2-2) If the system dynamics f(x) in (2-1) is well known, there exists an ideal controller as [43]
e k e k e
k x f
u* =− (x)+ c(n) + n (n−1) +...+ 2&+ 1 (2-3) where k , i i=1,2,L,n is non-zero positive constant. Substituting (2-3) into (2-1) yields
0 ... 2 1
) 1 ( )
( +k e − + +k e+k e=
en n n & . (2-4) If k are chosen to correspond to the coefficients of a Hurwitz polynomial whose roots lie i strictly in the open left half of the complex plane, then lim =0
∞
→ e
t
can be inferred for any starting initial conditions. However, because the system dynamics f(x) may be unknown or perturbed in practice, the ideal control law u in (2-3) cannot be implemented easily. To *
solve the problem of the model-based control approach for real-time implementation, adaptive fuzzy neural-network control (AFNC) techniques have been developed to control these kinds of unknown nonlinear dynamic systems [11-14]. These techniques use a structure of FNN to estimate the plant or controller parameters in a real-time environment. If the FNN is applied to estimate the model of the plant, it is called an indirect AFNC, and if the FNN is applied to estimate the controller of the plant, it is called a direct AFNC [44].
sliding
asymmetric fuzzy neural network control
r
asymmetric fuzzy neural network control
r
Fig. 2-1. The block diagram of ASAFNC system.
According to the design concept of the direct AFNC, we propose an adaptive self-structuring asymmetric fuzzy neural-network control (ASAFNC) system as shown in Fig.
2-1. The ASAFNC system is composed of a SFNN controller and a robust controller as
rb sfnn
ac u u
u = + (2-5) The SFNN controller usfnn utilizes the SFNN with asymmetric Gaussian membership
functions to mimic the ideal controller in (2-3), and the robust controller u is designed to rb compensate for the modeling error between the SFNN controller usfnn and the ideal controller u . For further analysis, first define a sliding surface as *
τ d e k e k e
k e
s = n− + n n− + + 2 + 1
∫
0t )2 ( ) 1
( L . (2-6) Substituting (2-5) into (2-1) and using (2-3) and (2-6), yields
rb sfnn u u u
s&= *− − . (2-7)
2.2 Description of SFNN
Fuzzy logic and NNs are complementary technologies in the design of intelligent systems. FNNs retain the basic properties and functions of NNs with some of their elements being fuzzified. In this approach, a network’s domain knowledge becomes formalized in terms of fuzzy sets, later being applied to enhance the learning of the network and augment its interpretation capabilities. By incorporating fuzzy principles into a NN, more user flexibility is attained and the resultant network or system becomes more robust [4]. FNNs are generally a fuzzy inference system constructed from structure of NN. Learning algorithms are used to adjust the weightings of the fuzzy inference system.
Figure 2-2 shows the configuration of the proposed SFNN which is composed of the input, the membership, the rule, and the output layers. Layer 1 accepts the input variables.
Nodes at layer 2 are term nodes which act as membership functions to represent the terms of the respective linguistic variables. The asymmetric Gaussian membership function constituted by a center, a left-side variance, and a right-side variance is considered. Nodes of layer 3 are regarded as fuzzy rules. The links before layer 3 represent the preconditions of rules and the links after layer 3 represent the consequences. Layer 4 is the output layer, where the node in this layer is the output of the NN. The interactions for those layers are given as follows.
x1
cancel the r-th rule add a new rule
th r<P β
rule generating process
rule pruning process
βk
cancel the r-th rule add a new rule
th r<P β
rule generating process
rule pruning process
βk
Fig. 2-2. The structure of SFNN.
Layer 1 - Input layer: For every node i in this layer, the net input and the net output are represented as
i variables. They mean that output equals input in this layer. This layer of SFNN just executes the transmission work.
Layer 2 - Membership layer: In this layer, each node performs a membership function and acts as a unit of memory. The bell-shaped function is adopted as the membership function.
For the ith input, the corresponding net input and output of the jth node can be expressed as
( ) ( )
2 22 2 2 2
ij ij i ij
m net x
σ
− −
= (2-10)
(
ij) (
ij)
ij
ij f net net
y 2 2 2
2 = =exp , j=1 ,2,L,M (2-11) where 2mij is the mean, 2σij is the variance and M is the total number of membership functions with respect to the respective input node. In this study, the input linguistic variable is the tracking error vector.
Layer 3 - Rule layer: Each node k in this layer is denoted by ∏ which multiplies the incoming signals and outputs the result of the product. For the kth rule node, the operation of the net input and output of this layer is presented as
∏
= ij ij
k w x
net 3 3
3 (2-12)
(
k)
kk
k f net net
y 3 3 3
3 = = , k =1 ,2,L,N (2-13) where 3xij represents the i, th input to the kth node of layer 3, j 3wij between the membership and the rule layers are assumed as unity, and N is the total number of fuzzy rules.
Layer 4 - Output layer: The single node o in this layer is labeled as Σ , which computes the overall output as the summation of all incoming signals. It executes the sun-of-weighting defuzzification. The description of the net input and output is expressed as
k k k
o w x
net 4 4
4 =∑ (2-14)
(
o)
oo
o f net net
y 4 4 4
4 = = (2-15) where 4wk is the output action strength of the output associated with the kth rule, 4xk represents the kth input to the node of layer 4, and 4yo is the output of SFNN.
In order to improve the learning capability and flexibility of a NN, asymmetric Gaussian membership functions are adopted, instead of ball-shaped functions described in layer 2.
According to the above description, the output of the SFNN with N existing fuzzy rules can be represented simply as
∑
== N
k k k
o w
y
1
)
φ (x (2-16) in which w is the output action strength associated with the k-th rule and k φk is the response of the firing weight for an input vector x=[x1x2LxL]T and composed of asymmetric Gaussian membership functions defined as [42]
( )
where M is the total number of membership functions with respect to the respective input node; m , ij σijl , and σijr are the mean, left-side variance, and right-side variance of the asymmetric Gaussian function in the j-th term of the i-th input linguistic variable x , i respectively. However, σ and ijl σ may become zero in the training procedure, the ijr membership function ζij will not be defined. To avoid this problem, this dissertation considers a membership function form as [44]
( )
where ϖ is a small positive constant. Then, the associated firing strength can be defined as
∏
=Thus, the output of the SFNN can be represented in a vector form as ) structure of the FNN should be determined in advance by empiricism. However, it is difficult to consider the balance between the rule number and the desired performance. Therefore, the structure adaptation algorithm which contains the growing and pruning of membership functions and fuzzy rules is proposed in this dissertation. The descriptions are given as follows.
In the process of the growing of membership functions, the concept which decides
whether to add a new node (membership function) in layer 2 and the associated fuzzy rule in layer 3 will be introduced. The mathematical description of the existing rules can be expressed as a set of clusters. For constructing the initial fuzzy rules of the SFNN, the fuzzy clustering method is used to partition a set of data into a number of overlapping clusters based on the distance in a metric space between the data points and the cluster prototypes. Each cluster in the product space of the input-output data represents a rule. The firing strength of a rule for each incoming data x can be represented as the degree that the incoming data i belong to the cluster [19]. If the value of firing strength is too small, it indicates that the input value is on the edge of range of the existing membership functions. Under this situation, the output will cause unsatisfactory performance. Therefore, a new membership function and a new fuzzy rule should be generated to improve the performance.
The firing strength from (2-19) is used as the degree measure
k
k φ
β = , k =1,2,...,N(t) (2-24) where )N(t is the number of the existing fuzzy rules at the time t. Define the maximum degree βmax as
t k N
k β
β
) (
max 1max
≤
= ≤ . (2-25) If βmax ≤Gth is satisfied, where Gth∈(0,1) is a pre-given threshold, the incoming data is far from the edge of range of the existing membership functions. Hence, a new membership function is generated. The mean and the variance of the new membership function and the weight are selected as follows
i new
i x
m = , (2-26)
i new l
i σ
σ , = , (2-27)
i new r
i σ
σ , = , (2-28)
=0
wnew (2-29)
where x is the new incoming data and i σi is a pre-specified constant. If the unknown control system dynamics is too complex, we can choose the larger Gth so that many membership functions can be created.
sampling time
Rise and decay curves of the used frequency index
sampling time
Rise and decay curves of the used frequency index
Fig. 2-3. The rise and decay curves of the used frequency index.
Next, to avoid the unrestricted growth of network structure and an overload computation, the pruning algorithm is developed to eliminate irrelevant fuzzy rules. In Ref. [21], a significance index is determined for the importance of the fuzzy rules. The elimination algorithm is derived from the observation that if the significance index fades when the firing weight is smaller than a special threshold value and if the significance index fixes when the firing weight is larger than a special threshold value [21]. In this dissertation, when the r-th firing strength βr is smaller than the threshold value P , it indicates that the relationship th becomes weak between the input and the r-th rule. Then, the significant index of r-th fuzzy rules will be decreased. When the r-th firing strength βr is larger than the threshold value P , it indicates that the incoming inputs fall into the range of the r-th fuzzy rule. Thus, the th
significant index of r-th fuzzy rules should be raised. The rise and decay curves of the used frequency index show in Fig. 2-3. The significance index is determined for the r-th rules can be given as
( )
( )
[ ]
⎪⎩
⎪⎨
⎧
≥
−
−
−
⋅
<
−
= ⋅ +
th r r
r
th r r
r I t I t if P
P if
t t I
I τ β
β τ
, ) ( 1 exp 2 ) (
),
exp(
) ) (
1 (
2
1 , )r=1,2,L,N(t (2-30)
where Ir is the significant index of the r-th rule and its initial value is 1, P is the pruning th
threshold value, and τ1 and τ2 are the designed constant. Exponential functions in (2-30) are used to rise or decrease the values of significant index in [0, 1]. If Ir ≤Ith is satisfied, where I is another pre-given threshold, the r-th fuzzy rule will be deleted. For real-time th implementation, if the computation load is the issue having highest priority, P should be th chosen large, so that more fuzzy rules can be pruned. This operation will prevent the fuzzy rule, which may be less used but still significant, from being deleted in the training process.
Hence, the computation load would be reduced.
In summary, the flow chart of the structure learning algorithm is shown in Fig. 2-4. The major contributions of the SFNN are: 1) SFNN can be operated directly without spending much time pre-determining membership functions and fuzzy rules; and 2) the computation load can be reduced simultaneously.
start
calculate the tracking error and sliding surface
max ≤Gth?
grow a membership function and fuzzy
rule
calculate the adaptive law and update the parameter
calculate the
prune the r-th membership function and rule
end control ?
End Yes
No calculate the
robust control
apply control law into system
calculate the tracking error and sliding surface
max ≤Gth?
grow a membership function and fuzzy
rule
calculate the adaptive law and update the parameter
calculate the
prune the r-th membership function and rule
end control ?
End Yes
No calculate the
robust control
apply control law into system
Fig. 2-4. The flow chart of the ASAFNC system.
2.3 Approximation of SFNN
An optimal SFNN controller can be designed to approximate the ideal controller (2-3) even under the structural change of neural network, such that [44, 45]
∆
σ are the optimal vectors. In fact, the optimal vectors that best approximate a given r
nonlinear function are difficult to be determined. Thus, an estimated SFNN controller is introduced as
φ
σ , and l σ , respectively. Moreover, the optimal vectors can be further defined as [44] r
⎥⎦⎤
D are positive constants specified by designers. There exists
∆* which is a finite positive constant such that the inequality ∆ ≤∆* can be held. Define a technique is employed to transform the nonlinear fuzzy function into a partially linear form so that the expansion φ~ can be expressed as [46]
h Substituting (2-39) into (2-38), (2-38) can be rewritten as
∆
Next, the uncertain term ε is satisfied
∆
*
2.4 ASAFNC Design
By using (2-40), (2-7) can be rewritten as
rb prescribed attenuation constant. If the system starts with initial conditions s(0)=0 ,
0 ) 0
~( =
w , 0m~(0)= , ~σl(0)=0, and ~σr(0)=0, the L2 tracking performance in (2-48) can be rewritten as
ε δ error tracking control without disturbance attenuation. To determine the adaptive laws of the parameters of ASAFNC appropriately and guarantee the closed-loop system stability, the Lyapunov function candidate is defined as
r
Differentiating (2-50) with respect to time and using (2-47) yield
r
Choose the adaptive laws as
φ
and the robust controller is designed as
s Thus, equation (2-51) can be rewritten as
2 )
Since 0V(t)≥ , we can arrange (2-58) as follows then the sliding surface s will converge to a certain small boundary. It is implied that the tracking error e will also converge to a certain small boundary [47].
2.5 Boundary Analysis Using Projection Algorithm
Although the stability of ASAFNC can be guaranteed, the parameters wˆ , mˆ , σˆ , and l σˆ cannot be guaranteed within a desired bound value by using the adaptive laws r
(2-52)-(2-55). According to the projection algorithm [44, 48, 49], the adaptive laws can be modified as follows. The adaptive law of weight is
⎩⎨
where the projection operator is given as
w w
The adaptive law of mean of asymmetric membership function is
⎩⎨
where the projection operator is given as
m m
The adaptive law of left-side variance of asymmetric membership function is
⎪⎩
where the projection operator is given as
l
The adaptive law of right-side variance of asymmetric membership function is
⎪⎩
where the projection operator is given as
r
Thus, the fact that the uncertain term ε is bounded can be guaranteed by the modified adaptive laws (2-60), (2-62), (2-64), and (2-66). The following description states that the analytic result of stability is the same as (2-59) by re-selecting the adaptive laws (2-60), (2-62), (2-64), and (2-66). First, define some useful variables as
~)
Then, the derivative of Lyapunov function shown in (2-51) can be rewritten as )
ˆ 0
can be obtained. Hence, for any possible condition occurs in (2-60), (2-62), (2-64), and (2-66), the conditions Jw ≤0, Jm ≤0, ≤0
σl
J , and ≤0
σr
J can be satisfied. Then, (2-72) can be reorganized as
) By substituting the robust controller (2-56), (2-78) can be rewritten as
2 ) Using the same discussion in the section 2.4, the stability of the system with the projection algorithm can also be guaranteed.
2.6 Simulation Results
In this section, the proposed ASAFNC is applied to a second-order chaotic dynamics system to verify its effectiveness. This scheme emphasizes that the parameter and network structure of the SFNN can be tuned online by the proposed algorithm. Consider a second-order chaotic dynamics system such as the Duffing’s equation describing a special nonlinear circuit or a pendulum moving in a viscous medium as follows [46]
u f
x&&= (x)+ (2-80) where )f(x)=−px&−p1x− p2x3 +qcos(ωt is the system dynamics, t is the time variable, ω is the frequency, u is the control force, and p , p1, p2, and q are real constants. The solutions of (2-80) may exhibit periodic depending on the choice of these constants, i.e., it is almost periodic and chaotic behavior. The open-loop system behavior, i.e., u =0, is simulated with p=0.4 , p1 =−1.1, p2 =1.0, and ω =1.8 for observing the chaotic unpredictable behavior. The phase plane plots with an initial condition point (0, 0) are shown in Figs. 2-5(a) and 2-5(b) for q=1.95 and q=7.00, respectively. The uncontrolled chaotic system has different trajectories for different values of q. To illustrate the effectiveness of the proposed design method, a comparison among a fix-structure AFNC using symmetric Gaussian membership functions [50], a fix-structure AFNC using asymmetric Gaussian membership functions [51], and the proposed ASAFNC is made.
q=1.95
(a)
x x &
q=1.95
(a)
x
x &x&
q=7.00
(b) x
x &
q=7.00
(b) x
x &x&
Fig. 2-5. Phase plane of uncontrolled chaotic dynamics system.
2.6.1 Comparison with AFNC
The simulation results of fix-structure AFNC using 3 symmetric membership functions are shown in Fig. 2-6. The tracking responses of state x are shown in Figs. 2-6(a) and 2-6(d);
the tracking responses of state x& are shown in Figs. 2-6(b) and 2-6(e); and the associated control efforts are shown Figs. 2-6(c) and 2-6(f) for q=1.95 and q=7.00, respectively.
The simulation results show that the tracking responses decline when membership functions are selected insufficiently.
xc
x
time (sec) (a)
state, x
xc
x
time (sec) (a)
state, x
x&
time (sec) (b) x&c
state, x&
x&
time (sec) (b) x&c
state, x&x&
time (sec) (c)
control effort
time (sec) (c)
control effort
xc
x
time (sec) (d)
state, x
xc
x
time (sec) (d)
state, x
x&
time (sec) (e) x&c
state, x&
x&
time (sec) (e) x&c
state, x&x&
time (sec) (f)
control effort
time (sec) (f)
control effort
Fig. 2-6. Simulation results of AFNC using 3 symmetric membership functions.
Next, the simulation results of fix-structure AFNC using 20 symmetric membership functions are shown in Fig. 2-7. The tracking responses of state x are shown in Figs. 2-7(a) and 2-7(d); the tracking responses of state x& are shown in Figs. 2-7(b) and 2-7(e); and the associated control efforts are shown Figs. 2-7(c) and 2-7(f) for q=1.95 and q=7.00, respectively. The simulation results show that the favorable tracking performance can achieve;
however, the computation load is heavy. These results demonstrate the fact that it is difficult to consider the balance between the rule number and the desired performance.
xc
x
time (sec) (a)
state, x
xc
x
time (sec) (a)
state, x
x&
time (sec) (b) x&c
state, x&
x&
time (sec) (b) x&c
state, x&state, x&x&
time (sec) (c)
control effort
time (sec) (c)
control effort
xc
x
time (sec) (d)
state, x
xc
x
time (sec) (d)
state, x
x&
time (sec) (e) x&c
state, x&
x&
time (sec) (e) x&c
state, x&state, x&x&
time (sec) (f)
control effort
time (sec) (f)
control effort
Fig. 2-7. Simulation results of AFNC using 20 symmetric membership functions.
To show that the learning capability of neural network can be upgraded as using the asymmetric Gaussian membership functions, the fix-structure AFNC using asymmetric Gaussian membership functions is applied to chaotic dynamics system again. The simulation results of fix-structure AFNC using 3 asymmetric membership functions are shown in Fig.
2-8. The tracking responses of state x are shown in Figs. 2-8(a) and 2-8(d); the tracking responses of state x& are shown in Figs. 2-8(b) and 2-8(e); and the associated control efforts are shown Figs. 2-8(c) and 2-8(f) for q=1.95 and q=7.00, respectively. The simulation results show that the favorable tracking performance can be achieved.
xc
x
time (sec) (a)
state, x
xc
x
time (sec) (a)
state, x
x&
time (sec) (b) x&c
state, x&
x&
time (sec) (b) x&c
state, x&state, x&x&
state, x&state, x&x&