High-Order Hopfield-based Neural Network for
Nonlinear System Identification
Chi-Hsu Wang*, IEEE Fellow
Department of Electrical and Control Engineering, National Chiao-Tung University,
Hsin Chu, Taiwan. 300 E-mails: [email protected] *Corresponding Author: Chi-Hsu Wang;
Kun-Neng Hung
Department of Electrical and Control Engineering, National Chiao-Tung University,
Hsin Chu, Taiwan. 300 E-mails: [email protected]
Abstract—The high-order Hopfield neural network
(HOHNN) with functional link net has been developed in this paper for the purpose of system identification of nonlinear dynamical system. The weighting factors in HOHNN will be tuned via the Lyapunov stability criterion to guarantee the convergence performance of real-time system identification. In comparison with the traditional Hopfield neural network (HNN), the proposed architecture of HOHNN has additional inputs for each neuron which has the advantages of faster convergence rate and less computational load. The simulation results for both HNN and HOHNN are finally conducted to show the effectiveness of HOHNN in system identification of uncertain dynamical systems. It is obvious from the simulation results that the performance of system identification for HOHNN is better than that of HNN.
Keywords—Hopfield neural network, functional link net,
Lyapunov theorem.
I. INTRODUCTION
The Hopfield neural network (HNN) was proposed in 1982 [1], which consists of a neuron set with feedback structure, is an auto-associative learning network. The HNN forms a multiple-loop feedback system where the number of feedback loops is equal to the number of neurons. The HNN has been adopted for pattern recognition [2], adaptive control [3], and crossbar switching problem [4], in recent years. Neural networks are suitable for identifying nonlinear dynamic systems because of its learning and memorizing capabilities, like HNN.
Functional link net methodology was first proposed in 1989 [5, 6] which have been combined with the neural network to create the high-order neural networks (HONNs). The input pattern of a functional link net is either a functional expansion representation, tensor representation or a combination of these two representations. The effectiveness of the functional link representation in classification was first demonstrated in [5, 6]. It has shown that the functional link net may be conveniently used for functional approximation and pattern classification with faster convergence rate and less computational load than a multi-layer perceptron (MLP) structure. There have been many considerable interests in exploring the applications of functional link model to deal with nonlinearity and uncertainties such as recognition of English language script [7], signal enhancement [8], the determination of transport modes [9], and the design of robot controller [10]. The advantages of high-order functional link net have been
shown in [11], in which not only the efficiency of supervised learning is greatly improved, but also a flat net without hidden layer is capable enough to do the job. However the functional link neural networks in [12] does not include the Hopfield-based neural network (HNN), and the weighting factors were tuned by back-propagation algorithm which can not guarantee the convergence of tuning results, especially in the real-time applications.
In this paper, a new high-order Hopfield-based neural network (HOHNN) is proposed. It is basically a HNN with the compact functional link net whose exact analytical expression is also proposed in this paper. The application of HOHNN in the identification is explored in this paper to show the advantages of extra inputs for each neuron in HOHNN. The existence of an optimal weighting matrix in HOHNN for the identification of unknown nonlinear dynamical system is first proposed. A Lyapunov-based tuning algorithm is then proposed to find the optimal weighting matrix of HOHNN to achieve favorable approximation error which can be attenuated to arbitrary specified level. The simulation results for both HNN and HOHNN are finally conducted to show the effectiveness of HOHNN in system identification of nonlinear dynamical system. It is obvious from the simulation results that the performance of system identification for HOHNN is better than that of HNN.
II. HOPFIELD-BASED NETWORK MODELS
There are three different types of artificial neural networks based on their feedback link connection architecture. In this paper, we focus attention on the recurrent Hopfield-based neural network. Considering the noiseless dynamic model of a single neuron in HNN illustrated in Fig. 1, the input vector
] ) ( ) ( ) ( [ ) (t x1 t x2 t x t x i n i i i !
= represent voltages, the
weighting vector w(t) [w1(t) w2(t) w (t)] i n i i i ! =
represent conductance, and n represents the number of neurons. The input vector xi(t) in Fig. 1 are fed back from
the output vector y(t)=[y1(t) y2(t) ! yn(t)], and a
current source Ii represents the externally applied bias.
Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics San Antonio, TX, USA - October 2009
Σ ) ( 1t xi w1(t)x1(t) i i i w1 ) ( 2t xi w2(t)x2(t) i i i w2 ) (t xi i i i w w(t)xi(t) i i i ) (t xi n ) ( ) (tx t w i n i n i n w ( )⋅ ϕ ¦ = n i i i i itxt w 1 ) ( ) ( ( )t yi i v Neural output i R i C Synaptic inputs Current source i I ΣΣ ) ( 1t xi w1(t)x1(t) i i i w1 ) ( 2t xi w2(t)x2(t) i i i w2 ) (t xi i i i w w(t)xi(t) i i i ) (t xi n ) ( ) (tx t w i n i n i n w ( )⋅ ϕ ¦ = n i i i i itxt w 1 ) ( ) ( ( )t yi i v Neural output i R i C Synaptic inputs Current source i I Figure 1. The ith
neuron in a Hopfield neural network.
The nonlinear function ϕ(⋅) is a sigmoid function which limits the permissible amplitude range of the sum of the inputs is defined by hyperbolic tangent function:
¸ ¹ · ¨ © § = 2 tanh ) ( i i i v a v ϕ (1) where ai is referred to as the gain of ith neuron. By the Kirchhoff’s current law, the following dynamic node equation can be obtained:
i n i i i i i i i w t x t I R t v dt t dv C + =
¦
+ =1 ) ( ) ( ) ( ) ( , i=1,",n (2)Because the input is the feedback of the combination of the output, (2) becomes
i n i i i i i i i w t v t I R t v dt t dv C + =
¦
+ =1 )) ( ( ) ( ) ( ) ( ϕ , i=1,",n. (3)The stability analysis of the above HNN has been shown in [13], in which an energy function was defined and the derivative of the energy function can be shown to be negative to yield an asymptotical stable system.
III.THE COMPACT FUNCTIONAL LINK NET WITH
HIGH–ORDER HNN (HOHNN)
The functional link net is a single-layer network in which the need for hidden layer is removed. However, such functional transforms increase the number of components greatly as the dimensions of input vector increases. Hence, it was suggested in [5] that high-order terms beyond the second order are not required in the enhanced patterns of input vector by omitting the terms with two or more equal indices. Therefore, a compact functional link net can be defined with rigorous formulae as shown in the following Fig. 2 and equations.
] [z1 z2 " zN
=
Z Input pattern
Functional link net 1 z z2 " zN z1z2 " zN−1zN " " 1 w w2 " N " w wN+1 wN(N+1)/2 Enhanced pattern v Output node Weighting factor ] [z1 z2 " zN = Z Input pattern
Functional link net 1 z z2 " zN z1z2 " zN−1zN " " 1 w w2 " N " w wN+1 wN(N+1)/2 Enhanced pattern v Output node Weighting factor
Figure 2. A compact structure of functional link net.
The input pattern vector Z in Fig. 2 can be defined precisely as follows: ( )» ¼ º « ¬ ª = + + 2 1 1 2 1 z zN zN zNN z " " Z (4) and
{
}
{
}
. ) 1 ( ,..., 2 , 1 | ,..., 2 , 1 | 2 ,..., 2 , 1 | 1 ,..., 2 , 1 | 1 1 1 2 ) 2 )( 1 ( ) 1 ( 2 ) 1 ( 2 2 2 ) 1 2 ( 1 1 1 1 1 4 2 2 1 1 °¿ ° ¾ ½ °¯ ° ® − − = = °¿ ° ¾ ½ °¯ ° ® − = = − = = − = = − − + − + » ¼ º « ¬ ª − − − − + + » ¼ º « ¬ ª − − + + − + + − − N N z z z k N z z z N z z z N z z z N N N N N N N k k k k k kN N N N N k A " A " A A A A A A A A A A (5)Equation (4) says that the Z vector has N (N + 1) / 2 terms, in which N is the number of original input variables. Also (5) describes all the extra second order terms for
( ) ¿ ¾ ½ ¯ ® + + + 2 1 2 1 N NN N z z
z " . Further we can let the input vector
of HNN shown in Fig. 1 to be a compact functional link net defined in Fig. 2. This combination in the HOHNN for a single neuron is shown in the following Fig. 3.
Σ ϕ( )⋅ yi( )t i v i R i C Functional Link Net 2 z 2 / ) 1 )( (n+Mn+M+ z n z 1 + n z M n z+ 2 + +M n z 1 z 2 + n z 1 + +M n z » » » » ¼ º « « « « ¬ ª ⋅ ⋅ ⋅ = − M M h u u y y y y 1 3 1 2 1 ) ( ) ( ) ( ) ( # ϕ ϕ ϕ ϕ Z ϕ w u w h w U ĭ U ĭ ΣΣ ϕ( )⋅ yi( )t i v i R i C Functional Link Net 2 z 2 / ) 1 )( (n+Mn+M+ z n z 1 + n z M n z+ 2 + +M n z 1 z 2 + n z 1 + +M n z » » » » ¼ º « « « « ¬ ª ⋅ ⋅ ⋅ = − M M h u u y y y y 1 3 1 2 1 ) ( ) ( ) ( ) ( # ϕ ϕ ϕ ϕ Z ϕ w u w h w U ĭ U ĭ
Figure 3. The high-order HNN (HOHNN) for a single neuron.
The input pattern of HOHNN produced from functional link net is the enhanced pattern in which the neural feedback, control voltage input, and the combination of these two inputs are contained. In Fig. 3, ĭ is the
n-dimension vector of the network feedback, U is the M-dimension vector of the control force; and
h
Z is the
high-order term vectors to the system. For this compact functional link net, the dimension of input pattern has been expanded to N = (n + M) (n + M + 1) / 2. Thus, the
N-dimension input vector Z and the n × N matrix of
weighting factor matrix can be defined as
[
]
T h Z U ĭ Z= (6) and ] [w wu wh W= ϕ (7) where ϕw , wu, and wh represent the weighting factors
of feedback input, control voltage input and high-order term, respectively.
IV.THE FUNCTION APPROXIMATION USING HOHNN
The aim of this section is to discuss the capability of function approximation of HOHNN for unknown nonlinear
systems. Consider a continuous-time nonlinear dynamic system of the following form:
) , ( ux F x= (8) where T n x x x ] [ 1 2 " =
x is the system state vector
assumed to be available for measurement and the nonlinear function F( ux, ) describes the system dynamics. In
addition, a BIBO (Bounded Input, Bounded Output) condition is also imposed for (8). The closed-loop diagram of function approximation of HOHNN is shown in Fig. 4.
Unknown Nonlinear System Plant U Functional-link Net x x ˆ − + ( )xˆ ϕ e
High-order Hopfield-based Neural Network
Σ 1 , i w 2 , i w 1 ,n+ i w 2 /)1 )( (,+nMn+M+ i w ( )⋅ ϕ yi( )t i v i R i C n i w, 2 ,n+ i w M n i w + , 1 ,n+M+ i w 2 ,+ M+ n i w Unknown Nonlinear System Plant U Functional-link Net x x ˆ − + ( )xˆ ϕ e
High-order Hopfield-based Neural Network
Σ 1 , i w 2 , i w 1 ,n+ i w 2 /)1 )( (,+nMn+M+ i w ( )⋅ ϕ yi( )t i v i R i C n i w, 2 ,n+ i w M n i w + , 1 ,n+M+ i w 2 ,+ M+ n i w ΣΣ 1 , i w 2 , i w 1 ,n+ i w 2 /)1 )( (,+nMn+M+ i w ( )⋅ ϕ yi( )t i v i R i C n i w, 2 ,n+ i w M n i w + , 1 ,n+M+ i w 2 ,+ M+ n i w
Figure 4. The closed-loop configuration of HOHNN for function approximation.
In order to approximate the unknown nonlinear system, a HOHNN with single layer, fully connection, recurrent nets and functional link model is proposed in Fig. 4. Assuming that we need n neurons to identify an nth order unknown nonlinear dynamical system and from (3), the mathematical model of the proposed HOHNN for system identification with zero bias can be expressed as follows,
BWZ x A u x F xˆ= (ˆ, )≈ ˆ+ (9) where
[
]
T n x x xˆ ˆ ˆ ˆ= 1 2 " x =[
v1 v2 " vn]
T,[
RC RC RnCn]
diag−1/ 1 1 −1/ 2 2 −1/ = " A , and[
C C Cn]
diag1/ 1 1/ 2 " 1/ = B .Substituting (6) and (7) into (9), we have the following equation: h h uU Bw Z Bw ĭ Bw x A xˆ= ˆ+ ϕ + + . (10) where
[
]
T n x x xˆ) (ˆ ) (ˆ ) ( 1 ϕ 2 ϕ ϕ " = ĭ , (11)[
]
T M u u u1 2 " = U , (12) and[
]
T M M n h=ϕ(xˆ1)⋅ϕ(xˆ2) " ϕ(xˆ1)⋅ϕ(xˆ ) " " u −1⋅u Z . (13)In (11), ϕ(⋅) is a nonlinear sigmoid function. The input vector U is defined in (12). The Zh vector in (13) comes from (5) in the compact functional link net shown in Fig. 2. The function approximation problem consists of determining whether by allowing enough high-order connections with weighting matrix W such that the HOHNN model approximates the input-output behaviour of
an arbitrary unknown nonlinear dynamical system. Assume that there exists optimal weighting matrix
] [ * * * * h u w w wϕ =
W which can approximate the
nonlinear dynamical system to any degree of accuracy. Therefore, (8) can be completely described by a HOHNN using the following optimal form:
h h u U Bw Z Bw ĭ Bw Ax x= + ϕ* + * + * (14) Moreover, the optimal matrices can be further defined as [13, 14] ) , , (wϕ*wu*wh* » ¼ º « ¬ ª + + + − = ∈ ∈ ∈ ∈ ∈ ∈ ) ˆ ( ) , ( sup min arg , ˆ , , , ˆ h h u u u h h u u Z Bw U Bw ĭ Bw x A u x F ȍ ȍ x ȍ x ȍ w ȍ w ȍ w w w w x x ϕ ϕ ϕ (15) where } ) ( : { ϕ ϕ ϕ ϕ ϕ w w w w w ȍ = tr T ≤D (16) } ) ( : { u u tr u D T u u w w w w w ȍ = ≤ (17) } ) ( : { h h tr h D T h h w w w w w ȍ = ≤ . (18) where ϕ w D , u Dw and h
Dw can been seen as bounded in a
ball of radius to avoid the arbitrarily large weight values;
x
ȍ , ȍxˆ and ȍu are compact constraint sets for wϕ,
u
w and wh, respectively, specified by designers. Based on the above assumptions, the following result can be obtained.
A. Approximation properties of HOHNN
Suppose the system in (9) is initial at x(0)=xˆ(0), the
dynamic behavior of the HOHNN model is described by
) , ( ux BWZ Ax x= + (19) By adding and subtracting Aˆx, the system (9) is
rewritten as ) , ˆ ( ˆ ˆ Ax Gxu x= + (20)
where G(xˆ,u)=F(xˆ,u)−Axˆ. For e(0) = 0, the state error x
x
e= −ˆ satisfies the following differential equation
) , ˆ ( ) , (xu Gxu BWZ Ae e= + − (21) Since Z is a continuous function which satisfies a Lipschitz condition in the compact domain
ε
S , there exists
a constant k such that for all
ε S ∈ ) , ( ), , (x1 u x2 u , 2 1 2 1, ) ( , ) (x u Zx u x x Z − ≤k − . (22)
As shown in [15], the input preprocess via expansion does not affect the ability of network to approximate the nonlinear function. Therefore, it can be shown that if N is large enough then there exist weight matrices *
W W= such that δ ε ≤ − ∈ ) , ˆ ( ) , ˆ ( sup * ) , ˆ ( u x G u x Z BW u x S (23) where δ is a constant to be chosen later. The solution to
the differential (21) can be expressed as
(
)
(
)
³
− = t t− d e t 0 * * ) ( ] ) ( ), ( ˆ ) ( ), ( [ ) ( τ BWZxτ uτ BWZxτ uτ τ e A(
) (
)
³
− + t t− d e 0 * ) ( ] ) ( ), ( ˆ ) ( ), ( ˆ [ τ τ τ τ τ τ BW Zx u Gx u A . (24)Since A is a diagonal stability matrix, there exists a positive constant α such that t t
e eA ≤ −α and N < α * W k
= . Define ε is the required degree of approximation and based on these definitions of the constants α , N, the degree of accuracy δ in (23) can be chosen as
0
2 >
=εα −α
δ e N . (25)
Taking norms on both sides of (24) and using (22), (23), we have the following inequality
(
)
(
)
³
⋅ − ≤ t t− d e t e 0 * * ) ( ) ( ), ( ˆ ) ( ), ( ) ( A τ BWZxτ uτ BWZxτ uτ τ(
) (
)
³
⋅ − + t t− d e 0 * ) ( ) ( ), ( ˆ ) ( ), ( ˆτ τ τ τ τ τ BWZx u Gx u A³
− − − + ≤ t t L d e e N e 0 ) ( ) ( 2 τ τ ε α α τ .Using the Bellman-Gronwall Lemma [16], we obtain
³
− − − + ≤ N t t N d e e e t e 0 ) ( 2 ) ( ε α α τ τ 2 ε ≤ . (26) Thus, for any ε>0 and any finite T>0, there exist apositive integer N and an optimal weighting matrix *
W
such that the state x(t) with N high-order connections and weight values * W W= satisfies ε ≤ − ≤ ≤ ) ( ˆ ) ( sup 0 t t T t x x . (27)
The following illustrated example is the magnetic levitation system [17] which will be shown in Section VI as a benchmark example for the identification via HOHNN. The magnetic levitation system shown in Fig. 5 is a second-order nonlinear, unstable, and non-affine dynamical system which will be identified by the HOHNN.
) (t
u
Feedback sensor A ball of magnetic material Current
control
) (t
x
A pair of horizontal fixture
Electromagnet )
(t
u
Feedback sensor A ball of magnetic material Current
control
) (t
x
A pair of horizontal fixture
Electromagnet
Figure 5. The magnetic levitation system.
In Fig. 5, the basic components with levitate mass, used in gyroscopes, accelerometers, and fast trains are constructed. A pair of horizontal fixture in this system is set to balance the ball of magnetic material in the horizontal plane. The current input decide the magnetic force to control the vertical position of the ball, and the information of distance and velocity of the ball can be read from the
feedback sensor. The overall detailed structure for system identification can be shown in the following Fig. 6.
Σ 2 3 w ) (⋅ ϕ ˆx2 2 v R C 2 2 w 2 4 w 2 5 w 2 1 w 2 6 w Σ 1 3 w ) (⋅ ϕ ˆx1 1 v R C 1 2 w 1 4 w 1 5 w 1 1 w 1 6 w
Magnetic levitation system
− − 2 x 1 x + + 2 e 1 e u Functional-link net » » » » » » » » » » » » » » ¼ º « « « « « « « « « « « « « « ¬ ª = u x u x x x u x x 2 1 2 1 2 1 ˆ ˆ ˆ ˆ ˆ ˆ Z ΣΣ 2 3 w ) (⋅ ϕ ˆx2 2 v R C 2 2 w 2 4 w 2 5 w 2 1 w 2 6 w ΣΣ 1 3 w ) (⋅ ϕ ˆx1 1 v R C 1 2 w 1 4 w 1 5 w 1 1 w 1 6 w
Magnetic levitation system
− − 2 x 1 x + + 2 e 1 e u Functional-link net » » » » » » » » » » » » » » ¼ º « « « « « « « « « « « « « « ¬ ª = u x u x x x u x x 2 1 2 1 2 1 ˆ ˆ ˆ ˆ ˆ ˆ Z
Figure 6. The overall identification diagram of magnetic levitation system.
In Fig. 6, there exist two neurons and one control input for the HOHNN. The three inputs ˆx1, ˆx2, and u are combined to form the full input vector
{
xˆ1 xˆ2 u xˆ1xˆ2 xˆ1u xˆ2u}
=
Z which is fed into the HNN.
Since the magnetic levitation system shown in Fig. 5 satisfies the sense of BIBO stability, there exists an optimal matrix *
W in HOHNN for the identification purpose in Fig. 6. However, the true weighting matrix *
W will have to be found based on the Lyapunov criterion
V. THE LYAPUNOV-BASED TUNING OF HOHNN
The optimal weighting factors are difficult to be determined in fact, thus the adaptive laws for weighting factors training have to be appropriately designed to guarantee the approximation performance. Define the state error as e=x−xˆ, the derivative of e with respect to time can be obtained by (10) and (20)
h h uU Bw Z w B ĭ w B Ae e= + ~ϕ + ~ + ~ (28) where w~ϕ=wϕ*−wϕ, wu=wu −wu * ~ and h h h w w w~ = *− .
To find the weight adaptive laws that guarantee to minimize the approximation error for the approximation process, this problem can be obtained by proving the stability of the approximation system. The following theorem is to discuss the system stability and determine the adaptive laws of weighting factors based on no modelling error.
Theorem 1: A nonlinear dynamic system is considered in
(8) assumed to be modeled exactly by (14) and the approximation system is designed as (9). If the adaptive law of weighting factors in ith neuron are chosen as
i ii ii j i j q v b pe wϕ, =−ηϕ ϕ( ) , forj=1,2,!,n, (29) i ii ii k u i k n u ub pe w ,(+ ) =−η , fork=1,2,!,M, (30) and i ii ii l h i l M n h zb pe w,(+ +) =−η , forl=1,2,!,(n+M)(n+M−1)/2 (31) where ii
respectively;
ϕ
η , η and u η are positive learning rates, h the stability of the overall identification scheme is guaranteed.
Proof:
Consider the Lyapunov candidate function as
) ~ ~ ( 2 1 ) ~ ~ ( 2 1 ) ~ ~ ( 2 1 2 1 h T h h u T u u T T tr tr tr V e Pe w w w w w w η η η ϕ ϕ ϕ + + + = (32)
where P > 0 is chosen to satisfy the Lyapunov equation and
Q P A
PA+ T =− . Taking the derivative of V with respect to
time and using (28) yields
) ~ ~ ( 1 ) ~ ~ ( 1 ) ~ ~ ( 1 ) ( 2 1 h T h h u T u u T T T tr tr tr V e Pe ePe w w w w w w η η η ϕ ϕ ϕ + + + + = Pe B w Z Pe B w U Pe B w ĭ Qe e hT T T h T T u T T T T T ~ ~ ~ ( 2 1 2 1 + + + − = ϕ ) ~ ~ ~ h h T u T T Z w PB e U w PB e ĭ w PB e + + + ϕ ) ~ ~ ( 1 ) ~ ~ ( 1 ) ~ ~ ( 1 h T h h u T u u T tr tr tr w w w w w w η η η ϕ ϕ ϕ + + + . Because ĭTw TBTPe ϕ ~ , U w TBTPe u T~ , Pe B w Z hT T T h ~ , ĭ w PB eT ~ϕ , e PBw U u T ~ , and h h TPBwZ
e ~ are all scales, the
following relationship can hold as
ĭ w PB e Pe B w ĭ Pe B w ĭT~ϕT T =( T~ϕT T )T = T ~ϕ , U w PB e Pe B w U Pe B w U T T T T u u T T T u T ~ ) ~ ( ~ = = , and h h T T T T h T h T T h T h w B Pe Z w B Pe e PBw Z Z ~ =( ~ ) = ~ .
Hence, the derivative of V with respect to time can be reorganized as Pe B w Z Pe B w U Pe B w ĭ Qe e hT T T h T T u T T T T T V ~ ~ ~ 2 1 + + + − = ϕ ) ~ ~ ( 1 ) ~ ~ ( 1 ) ~ ~ ( 1 h T h h u T u u T tr tr trw w w w w w η η η ϕ ϕ ϕ + + + . (33)
Here, select the following equations
Pe B w ĭ w w T T T T tr ϕ ϕ ϕ ϕ η (~ ~ ) ~ 1 − = , (34) Pe B w U w w T T u T u T u u tr(~ ~ ) ~ 1 − = η , (35) and Pe B w Z w w hT T T h h T h h tr(~ ~ ) ~ 1 − = η . (36)
Substituting (34)-(36) into (33), (33) can arrange as
0 2 1 ≤ − = eTQe V . (37)
Based on (34)-(36), the adaptive law of weighting factors (29)-(31) can be obtained in an element form. Thus, the Lyapunov stability theorem of overall approximation scheme is guaranteed under the optimal approximation model. Q.E.D.
VI.ILLUSTRATED EXAMPLES
From Section IV, the identification diagram of magnetic levitation system has been discussed. However, the design structure is not complete because there are no updates to the weighting factors. In this section, the adaptive law will be adopted in HOHNN and the parameters will be set in detail to the simulation. According to [17], the dynamic motion equation of the ball in Fig. 6 can be represented as:
2 1 x x = (38) 2 1 2 0 2 2 ) ( 2ma x au L g x m k x + − + − = (39)
where x1(m) is the vertical (downward) distance between the ball and the magnet, x2(m/second) is the vertical velocity of the ball, m = 0.1kg is the mass of the ball, k = 0.001N/m/s is a viscous friction coefficient, g is the acceleration of gravity, L0 = 0.02H is the nominal point inductance, a = 0.005m is positive constant, and u(A) is the current in the coil of the electromagnet. In order to obtain suitable data for identification, the control force is chosen as
(
() ())
())
(t k1x1t rt k2x2 u t
u =− − − + b (40) where k1=−50.9568 and k2=−2.5640 are the state feedback gains, r(t) is a reference position and ub(t) is a
model-based bias given by
(
)
0 2 ) ( 1 aL mg t r ub= + . (41) The reference in (41) is chosen to be a sum of five sinusoid functions with different frequencies plus an offset:¦
= + = 5 1 13 . 0 ) sin( 04 . 0 ) ( j jt w t r (42)where w1=0.5, w2=1, w3=3, w4=4, and w5=4. The parameters in HOHNN are selected as R = 100ȍ, C = 0.01F, and Q=diag
[
2 2]
. The initial voltages of two Hopfieldneurons are both zero voltages. The learning rates of weighting factors are selected as η = 0.08, ϕ η = 0.008, u and η = 0.001. The initial weighting factors h wϕ and wu
are set as 0.1 and wh is set as 0.001. The following simulation results show the approximation of magnetic levitation system using HNN and the proposed HOHNN. The command trajectories are xc1 and xc2 , and the identified vertical position and velocity of the ball are ˆx1
and ˆx2 . Fig. 7 and Fig. 8 show the behavior of approximation system using HNN and HOHNN. The detailed approximation errors of HNN and HOHNN are shown in Fig. 9 and Fig. 10. It is obvious from Figs. 9 and 10 that the approximation error is smaller for HOHNN, especially in the first 0.2 seconds.
0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 1 x 2 x 1 ˆ x 2 ˆ x 1 c x 2 c x Ti m e (se co n d) 0 0 .0 5 0 . 1 0 .1 5 0 . 2 0 . 2 5 0 .3 0 .3 5 0. 4 0 . 4 5 0 . 5 -2 -1 0 1 2 0 0 . 05 0 . 1 0 .1 5 0 . 2 0 .2 5 0 . 3 0 . 35 0 . 4 0. 45 0 .5 -2 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 1 x 2 x 1 ˆ x 2 ˆ x 1 c x 2 c x Ti m e (se co n d) 0 0 .0 5 0 . 1 0 .1 5 0 . 2 0 . 2 5 0 .3 0 .3 5 0. 4 0 . 4 5 0 . 5 -2 -1 0 1 2 0 0 . 05 0 . 1 0 .1 5 0 . 2 0 .2 5 0 . 3 0 . 35 0 . 4 0. 45 0 .5 -2 -1 0 1 2
Figure 7. Behavior of approximation system using HNN.
0 1 2 3 4 5 6 7 8 9 10 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 Ti m e (se con d) 1 x 2 x 1 ˆ x 2 ˆx 1 c x 2 c x 0 0 . 0 5 0. 1 0. 1 5 0. 2 0. 2 5 0. 3 0. 3 5 0. 4 0. 4 5 0 . 5 -2 -1 0 1 2 0 0 . 05 0 . 1 0.15 0 . 2 0.25 0 . 3 0 . 35 0 . 4 0 . 45 0 .5 -2 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -1 0 1 2 0 1 2 3 4 5 6 7 8 9 10 -2 -1 0 1 2 Ti m e (se con d) 1 x 2 x 1 ˆ x 2 ˆx 1 c x 2 c x 0 0 . 0 5 0. 1 0. 1 5 0. 2 0. 2 5 0. 3 0. 3 5 0. 4 0. 4 5 0 . 5 -2 -1 0 1 2 0 0 . 05 0 . 1 0.15 0 . 2 0.25 0 . 3 0 . 35 0 . 4 0 . 45 0 .5 -2 -1 0 1 2
Figure 8. Behavior of approximation system using HOHNN.
2 e 1 e Tim e (se co n d) Tim e (se co n d) 0 0.2 0.4 0.6 0.8 1 1.2 1 .4 1 .6 1.8 2 -2 -1.5 -1 -0.5 0 0.5 0 0.2 0.4 0.6 0.8 1 1.2 1 .4 1 .6 1.8 2 -2 -1.5 -1 -0.5 0 0.5 2 e 1 e Tim e (se co n d) Tim e (se co n d) 0 0.2 0.4 0.6 0.8 1 1.2 1 .4 1 .6 1.8 2 -2 -1.5 -1 -0.5 0 0.5 0 0.2 0.4 0.6 0.8 1 1.2 1 .4 1 .6 1.8 2 -2 -1.5 -1 -0.5 0 0.5
Figure 9. The approximation error of HNN.
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 -2 -1 .5 -1 -0 .5 0 0 .5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 -2 -1 .5 -1 -0 .5 0 0 .5 2 e 1 e Tim e (se co nd ) Tim e (se co nd ) 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 -2 -1 .5 -1 -0 .5 0 0 .5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 -2 -1 .5 -1 -0 .5 0 0 .5 2 e 1 e Tim e (se co nd ) Tim e (se co nd )
Figure 10. The approximation error of HOHNN.
VII. CONCLUSION
This paper has proposed a high-order Hopfield-based neural network (HOHNN) for the identification of unknown nonlinear dynamical systems. The approximation capability shows that HOHNN is capable of approximating the behavior of dynamical systems to any degree of accuracy. The optimal weighting matrices in HOHNN can be found via Lyapunov criteria. The adaptive laws to tune the weighting matrix can reduce the approximation error to any desired small level. The simulation results for HNN and HOHNN are finally conducted to show that the performance of system identification for HOHNN is better than that of HNN.
REFERENCES
[1] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proceedings of National Academy of Sciences, USA, vol. 79, pp. 2554-2558, April 1982.
[2] Donq Liang, Lee, “Pattern Sequence Recognition Using a Time-Varying Hopfield Network,” IEEE Trans. Neural Networks, vol. 13, no. 2, pp. 330-342, March 2002.
[3] Z. Ma, and A. Jutan, “Control of a pressure tank system using a decoupling control algorithm with a neural network adaptive scheme,” IEE Trans. Control
Theory, vol. 150, no. 4, pp. 389-400, July 2003.
[4] Yong Li, Zheng Tang, Guang-Pu Xia, and Rong-Long Wang, “A Positively Self-Feedbacked Hopfield Neural Network Architecture for Crossbar Swtiching,” IEEE Trans. Circuits and Systems-part I: Regular papers, vol. 52, no. 1, pp. 200-206, January 2005.
[5] Y. H. Pao, Adaptive pattern Recognition and Neural Networks, Addison-Wesley, Reading, 1989.
[6] Y. H. Pao, Functional-link net computing: theory, system architecture, and
functionalities, Computer, 1992.
[7] G. H. Park and Y. H. Pao, “Unconstrained word-based approach for offline script recognition using density-based random-vector functional-link net,”
Neurocomputing, vol. 31, no. 1-4, pp. 45-65, 2000.
[8] Bor-Shyh Lin, Bor-Shing Lin, Fok-Ching Chong and Feipei Lai, “A functional Link Network With Higher Order Statistics for Signal Enhancement,” IEEE
Trans. Signal Processing, vol. 54, no. 12, pp. 4821-4826, December 2006.
[9] S. M. Hosseini-Nezhad, T. S. Yamashita, R. A. Bielefeld, S. E. Krug and Y. H. Pao, “A neural network approach for the determination of interhosptial transport mode,” Comput. Biomed. Res., vol. 28, no. 4, pp. 319-334, 1995. [10] I. Hassanzadeh, S. Khanmohammadi, J. Jiang and G. Alizadeh,
“Implementation of a functional link net-ANFIS controller for a robot manipulator, in: Proceedings of the 3rd International Workshop on Robot Motion and Control,” Bukowy Dworek, Poland, pp. 399-404, 2002.
[11] Klassen, M. S. and Y. H. Pao, “Characteristics of the functional-link net: A higher order delta rule net,” IEEE Proceedings of 2nd Annual International
Conference on Neural Networks, San Diago, CA, June 1995.
[12] Jagdish C. Patra, Ranendra N. Pal, B. N. Chatterji, and Ganapati Panda, “Identification of nonlinear dynamic systems using functional link artificial neural networks,” IEEE Trans. System, Man, and Cybernetics-part B:
Cybernetics, vol. 29, no. 2, pp. 254-262, April 1999.
[13] Simon Haykin, Neural Networks, Upper Saddle River, NJ: Prentice-Hall, 1999. [14] E. B. Kosmatopoulos, M. M. Polycarpou, M. A. Christodoulou, and P. A.
Ioannou, “High-order neural network structures for identification for dynamical systems,” IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 422-431, Mar 1995. [15] N. E. Cotter, “The Stone-Weierstrass theorem and its application to neural
networks,” IEEE Trans. Neural Networks, vol. 1, no. 4, pp. 290-295, 1990. [16] J. K. Hale, Ordinary Differential Equations, New York: Wiley, 1969. [17] V. M. Becerra, F. R. Garces, S. J. Nasuto and W. Holderbaum, “An efficient
parameterization of dynamic neural networks for nonlinear system identification,” IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 983-988, July 2005.
