The linear-exponential-quadratic-Gaussian control for discrete systems with application to reliable stabilization

The linear-exponential-quadratic-Gaussian

control for discrete systems with application

to reliable stabilization

Der-Cherng Liaw

_{, Chun-Hone Chen}

Department of Electrical and Control Engineering, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu 30010, Taiwan, ROC

Abstract

In this paper, we derive the discrete linear-exponential-quadratic-Gaussian (LEQG) controller which can take both the system and measurement noise covariances into consideration. Comparing with the traditional linear-quadratic-Gaussian (LQG) design, the LEQG has the wilder design freedom. The proposed discrete LEQG control scheme is then applied to the study of reliable control which can tolerate abnormal operation within some pre-specified set of actuators. This is achieved by suitable modification of the algebraic Riccati equation for the design of the controller. The bounds of gain margins for the feedback control gains of reliable stabilization are also derived. The stability of the overall system is preserved despite the abnormal operation of actuators within a pre-specified subset in the bounds of gain margins.

Ó 2002 Elsevier Science Inc. All rights reserved.

Keywords: Reliable control; Discrete linear-exponential-quadratic-Gaussian control; Kalman filter; Algebraic Riccati equation

1. Introduction

The linear-exponential-quadratic-Gaussian (LEQG) method was recently studied for continuous-time systems [1,2], which is mainly based on the as-sumption of the estimated states approaching the true ones very quickly, and

*

Corresponding author.

E-mail address:[email protected](D.-C. Liaw).

0096-3003/02/$ - see front matter Ó 2002 Elsevier Science Inc. All rights reserved. PII: S 0 0 9 6 - 3 0 0 3 ( 0 2 ) 0 0 1 2 7 - 3

the true states in the performance index can then be replaced by the estimated ones. The corresponding state estimation procedure is similar to that of the traditional linear-quadratic-Gaussian (LQG) method using the Kalman filter, while the suboptimal control is also a linear combination of estimated states and can be obtained by solving the Hamilton–Jacobi–Bellman equation.

It is known that the control gain of LQG is deterministic since it is the same as the linear-quadratic-regulator (LQR) method where the LQR system is treated as deterministic. Thus, the LQG design has disadvantage of less re-sponsive in the environment with significant noise. In this paper, the proposed discrete LEQG method is intended to overcome this disadvantage since the design of the control gain will take both system and measurement noise co-variances into consideration. Although the derived LEQG design algorithm may be more complicated than that of the LQG method, it is more adaptive to a high-noise environment.

Reliable control is viewed as a means of ensuring system stability against the loss of control components. The basic elements of reliable control design are limited by the existing actuators of the physical system. This is somewhat different from redundant control which increases the number of actuators. In general, reliable control scheme divides the existing control components into two parts: the main controls and the auxiliary ones. For system stability, the main control part of actuators must never fail to operate. However, the aux-iliary control part can provide better system stability and performance. The reliable control system is required to be stable even if the auxiliary controls operate abnormally. Although the stability and performance of a reliable control system may not be better than those obtained by standard control de-sign, the reliable control system is guaranteed to be stable while the stan-dard control system may lose its stability when some specified control components function abnormally. This is the tradeoff between simplicity of system integration and risking system instability when actuators may operate abnormally.

In recent years, the design of reliable control laws has attracted considerable attention. The study of reliable linear-quadratic control of nonlinear systems by employing the Hamilton–Jacobi inequality in the nonlinear case has been proposed by Liaw and Liang [3,4]. The designed controllers were shown to be able to tolerate the malfunction within a pre-specified subset of actuators. The gain margin for guaranteeing system stability and the performance bound were estimated. In [5], Veillette proposed linear-quadratic (LQ) state-feedback reli-able control laws for continuous-time systems in which the actuator mal-function occurs within a pre-specified subset. In such a design, all the system states are assumed to be available for feedback design. This is generally not true for most practical applications. In many modern control systems, it is unusual to have all the states of a dynamical system available through mea-surements. Some system states may be impossible or too expensive to measure.

Thus, in this paper we will apply the LEQG control scheme to provide the stability of a discrete system in the presence of abnormal actuators. One of the major goals of this study is to solve the design algorithm for discrete LEQG control. The main concern is that the special structure of the LEQG controller explicitly takes both the system and measurement covariances into consider-ation while the LQG does not. The discrete LEQG control is then applied to the reliable control design.

The paper is organized as follows. In Section 2, the LEQG design is derived for general discrete-time systems. It is followed by the design procedure for the reliable LEQG control of discrete systems. An example is given in Section 4 to demonstrate the proposed design. Finally, Section 5 summarizes the main re-sults.

2. The LEQG control for discrete systems

Consider a class of linear discrete systems as given by

x_kþ1¼ Akxkþ Bkukþ Ckwk; ð1Þ

with measurement process

zk ¼ Hkxkþ vk; ð2Þ

where xk 2 Rn, uk 2 Rm and zk 2 Rl. Here, both wk and vk denote Gaussian

white noise with zero mean. Moreover, we assume that EfwkwTjg ¼ Wkdkj with

WkP0, EfvkvTjg ¼ Vkdkjwith Vk >0 and EfwkvTjg ¼ 0, where Efg denotes an

expectation function operator and dkj¼ 1 for k ¼ j while dkj¼ 0 elsewhere.

Before deriving the LEQG control law, the following definitions and notation are recalled and will be used in the paper.

Definitions and notations: Information set [6]:

Ik¼ fz0; . . . ; zk; u0; . . . ; uk 1g: ð3Þ

Induced information set:

Ik¼ f^xx0; . . . ; ^xxk; P0; . . . ; Pkg: ð4Þ

Priori state estimation:

xxk ¼ EfxkjIk 1g: ð5Þ

Posteriori state estimation: ^

Priori state estimation error covariance matrix: Mk ¼ Efðxk xxkÞðxk xxkÞ

T

g: ð7Þ

Posteriori state estimation error covariance matrix: Pk ¼ Efðxk ^xxkÞðxk ^xxkÞ

T

g: ð8Þ

Applying the separation theorem, a Kalman filter can be constructed to pro-duce the optimal estimated state from the noisy measurements. The state es-timation equation can be written as

^

xxk ¼ xxkþ Kksk; ð9Þ

where sk is defined as the innovation

sk ¼ zk Hkxxk; ð10Þ

with zero mean and covariance

Sk ¼ EðsksTkÞ ¼ HkMkHkTþ Vk: ð11Þ

From Eqs. (8)–(11), we have Pk ¼ ðI KkHkÞMkðI KkHkÞ

T

þ KkVkKkT: ð12Þ

Minimizing the trace of above estimation error covariance matrix Pk, i.e.,

traceðPkÞ, with respect to Kk [7], the Kalman gain is obtained as

Kk¼ MkHkTðHkMkHkTþ VkÞ 1: ð13Þ

Eq. (12) can then be rewritten as

Pk ¼ Mk MkHkTðHkMkHkTþ VkÞ 1HkMk: ð14Þ

From (5), we have

xxk ¼ Ak 1^xxk 1þ Bk 1uk 1: ð15Þ

Thus, the priori state estimation error covariance matrix Mk can be derived as

Mk ¼ Ak 1Pk 1ATk 1þ Ck 1Wk 1CTk 1: ð16Þ

By giving initial values ^xx0¼ xx0 and initial covariance matrix M0, we can

cal-culate the values for all Kalman gain Kk. The Kalman filter block diagram is

shown in Fig. 1.

The LEQG optimization problem for the discrete system (1) is to minimize the performance index:

PI¼ E exp l 2U h i

n o

where

U¼ ^xxT_NQNxx^Nþ

XN 1 k¼0

ð^xxT_kQk^xxkþ uTkRkukÞ; ð18Þ

with QN and Qk being positive semi-definite matrices, Rk a positive definite

matrix and l a tunable scalar.

From the definitions and derivations above, the LEQG control law can be obtained as in the following Algorithm. Note that, details of the derivation are given in Appendix A.

Algorithm (LEQG control law): If S 1

k lK T

kHkKk >0 for k¼ 0; . . . ; N , then the LEQG control law for the

minimization of performance index (17) can be obtained as

u_k ¼ Ck^xxk for k¼ 0; . . . ; N 1; ð19Þ where Ck¼ ðRkþ BTkKkþ1BkÞ 1BTkKkþ1Ak; ð20Þ Kk ¼ Hkþ lHkKkðS 1k lK T kHkKkÞ 1 K_kTHk ð21Þ and Hk 1¼ Qk 1þ ATk 1½Kk KkBk 1ðRk 1þ BTk 1KkBk 1Þ 1BTk 1Kk AN 1; ð22Þ with boundary condition HN ¼ QN.

Following the algorithm above, the performance index (17) can be simplified as J¼ a0exp l 2 xx T 0K0xx0 h i n o ; ð23Þ

where a0 is solved by the backward recursive process:

ak 1¼ akjI lKkTHkKkSkj 1=2; ð24Þ

with boundary condition aN ¼ 1.

Suppose system (1) is linear time-invariant with N ¼ 1 in (18). Employing an equality property of matrix inverses (see e.g., Appendix A.21 in [8]), Eqs. (20)–(22) can be rewritten as C¼ ðR þ BT_KBÞ 1_BT KA; ð25Þ where K¼ H þ lHKðS 1 _lKT HKÞ 1KTH¼ ðH 1 _lKSKT_Þ 1_{; ð26Þ} H¼ Q þ AT_{½K KBðR þ B}T_KBÞ 1_BT_{K A ¼ Q þ A}T_ðK 1_{þ BR} 1_BT_Þ 1_A ð27Þ and the steady Kalman gain is

K¼ MHT_ðHMHT_{þ V Þ} 1_{: ð28Þ}

Combining Eqs. (26) and (27), we have Q¼ H AT_ðH 1_{þ BR} 1_BT _lKSKT_Þ 1_A; ð29Þ i:e:; Q¼ H AT_D 1_{A lA}T_½D 1_KðS 1 _lKT_D 1_KÞKT_D 1 1_{A; ð30Þ} where D H 1þ BR 1BT_{: ð31Þ} Let QLEQG ¼ Q þ lAT½D 1KðS 1 lKTD 1KÞKTD 1 1A: ð32Þ Thus, we have QLEQG ¼ H ATD 1A¼ H AT½H HBðR þ BTHBÞ 1BTH A: ð33Þ

Let P be the algebraic Riccati equation (ARE) solution of the standard LQG problem with the same weighting matrices Q and R, i.e., P solves the ARE below:

Q¼ P AT_{½P PBðR þ B}T_PBÞ 1_BT_{P A: ð34Þ}

Then the LQG state-feedback control gain is known to be

Suppose l P 0. According to Lemma 4.1 of [10], we have Q 6 QLEQG which

implies P 6 H. Two properties of this LEQG design can then be summarized as follows.

Property 1. Suppose the LEQG solutions H and K satisfy Eqs. (26) and (27). Then P 6 H 6 K (resp. P P H P K) if the LQG solution P satisfies Eq. (34) with the same weighting matrices Q and R with l P 0 (resp. l 6 0).

Property 2. The LQG method is a special case of the LEQG control when l tends to zero, i.e., P ¼ H ¼ K.

It is observed from (29) that there exists a positive upper bound lmax such

that 0 6 BR 1_BT _lKSKT _{for l 6 l}

max. From Eqs. (25) and (35), Property 1

implies that the LEQG method in general has larger control gain than that of the LQG scheme. Moreover, with the larger control gain, the LEQG control system will have greater immunity to low-frequency environment noise. For simplicity and without loss of generality, we set the LEQG tunable scalar l as 0 6 l 6 lmax in the remainder of this paper.

The next result follows readily from Theorem 6.5 and Corollary 6.6 of [9], and the discussions above.

Lemma 1. Let Q¼ qT_{qP0, R > 0, CW C}T _{¼ FF}T_{P0 and V > 0. If ðA; BÞ is}

stabilizable, ðq; AÞ is detectable, ðH ; AÞ is detectable and ðA; F Þ is stabilizable, then system (1) is asymptotically stabilizable by LEQG control. The corre-sponding control gain C is given in Eq. (25) and estimator gain K is in Eq. (28).

3. LEQG scheme for reliable control design

In the following, we apply the LEQG control scheme derived in Section 2 to the stabilization of system (1) subject to the abnormal operation of actuators. Let the control matrix B and the weighting matrix R be decomposed as

B¼ BX0 BX   and R¼ RX0 0 0 RX 

respectively, where BX0corresponds to the normally operating actuators and B_X

for possible malfunctioning actuators.

First, we consider the worst case of which BX¼ 0, i.e., the minimum number

of actuators is under operation. Let the weighting matrices of the cost function in (17) be Q and RX0. Denote H the solution of the following ARE:

Q¼ H AT_ðH 1_{þ B}

X0R 1_X0BT_X0 lKSKTÞ 1A: ð36Þ

According to that of [11], there exists a unique and positive definite symmetric solution H for Eq. (36) ifðA; BX0Þ is stabilizable. Let

DX0  H 1þ B_X0R 1

X0B T

X0 lKSK

T_{: ð37Þ}

Since H > 0 and RX0 >0, we have D_X0 ¼ DT

X0 >0.

Next, we consider the reliable design of the estimator by checking whether the matrices H and K can also solve for the new ARE for the system (1) with BX6¼ 0. Let

Qrel H ATðH 1þ BR 1BT lKSKTÞ 1A: ð38Þ

From Eqs. (36) and (37) and an equality property of matrix inverses, we have

Qrel¼ Q þ AT½D 1X0B_XðR_Xþ BT_XD 1_X0B_XÞBT_XD 1_X0 A: ð39Þ

It is obvious that Qrel is a symmetric and semi-positive definite matrix. Thus

there exists a matrix qrel such that Qrel¼ qTrelqrel. In addition, from (37) and

(39) we have QrelP Q. According to Lemma 4.1 of [10], since QrelP Q, we have

thatðqrel; AÞ is a detectable pair if ðq; AÞ is a detectable pair too. The reliable

control gain can then be obtained as

C¼ ðR þ BT_KBÞ 1_BT KA ¼ RX0þ BT X0KBX0 BT X0KBX BT XKBX0 R_Xþ BT XKBX  1 BT X0 BT X0 " # KA; ð40Þ

which will also stabilize the closed-loop dynamics of system (1). We have the next theorem.

Theorem 1. Suppose the conditions of Lemma 1 hold. IfðA; BX0Þ is stabilizable,

then the closed-loop dynamics of linear discrete system (1) is asymptotically stabilizable in the presence of abnormal operation of actuators and satisfies the LEQG performance criterion as in (17), where Q is replaced by Qrelas defined in

(39). Moreover, the corresponding control gains C is given in Eq. (40). Now, we summarize the design procedure as follows:

Step 1: Suppose the conditions of Lemma 1 hold andðA; BX0Þ is stabilizable.

Solve for the solution H of the ARE (36) with parameters ðA; BX0;

Step 2: Substitute the solution H obtained from Step 1 to calculate K in Eq. (26) and the control gain as in (40) with parametersðA; B; Q; RÞ. Now, we consider the gain margin of the control gain C. Suppose R is a diagonal matrix and denote NC _{the diagonal matrix corresponding to the}

magnitude change of the control gain as given by

NC _¼ NX0 0

0 NX



¼ diagðnX01; nX02; . . . ; nX0r; nX1; nX2; . . . ; nXðm rÞÞ:

ð41Þ Multiplying C by NC_{, the closed-loop dynamics of (1) becomes x}

kþ1 ¼

ðA BNC_CÞx

k. From [12], A BNCCis stable ifðA BNCCÞ T

KðA BNC_CÞ

K 60, where K in (26) can be obtained by H from solving (36). This leads to the checking of the negative semi-definiteness forðA BNC_CÞT

KðA BNC_{CÞ K}

to provide the stability of A BNC_{C. Details are given in Appendix B. We then}

have the following result.

Theorem 2. The matrix A BNC_{C is stable if the gain matrix N}C _{satisfies either}

of the following two conditions:

ðiÞ 1 1þ a< nX0i< 1 1 a and bn 2 Xiþ c ð1 nXiÞ 2 dð1 nXiÞ 2 >0 or ðiiÞ 1 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi bþ c bc p 1 b < nXi< 1þpffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffibþ c bc 1 b and a2n2_X0_i ð1 n_X0_iÞ2 f ð1 n_X0_iÞ2>0;

where the scalars a, b, c, d and f are defined in Eqs. (B.10)–(B.14). Note that, A BNC_{Cis guaranteed to be stable if both n}

X0_i and n_Xistatisfy

the bounds of Theorem 2. However, if either nX0_i or n_Xi fail to satisfy the

bounds of Theorem 2, the eigenvalues of A BNC_{C must be calculated to}

check the system stability. According to the results of [13], we have a! 1, b! 1, c ! 1, d ! 0 and f ! 0 if the sampling time is sufficiently small. This leads to the unification of the two sufficient conditions of Theorem 2 as given in the next corollary.

Corollary 1. Suppose the sampling time is sufficiently small and the matrices R and NC _{are diagonal. Then A BN}C_{C is stable if (i)0:5 < n}

X0i<1 for all i ¼

Note that, Theorem 2 and Corollary 1 show the stability of A BðNC_CÞ

about the control gain variation. In fact, they can also be explained as the stability of A ðBNC_{ÞC with respect to the variation of control matrix B.}

4. Illustrative example

In this section, we present the numerical results for the proposed discrete-time LEQG reliable control. Suppose

A¼ 1 0:05 0 1  ð42Þ and B¼ BX0 BX   ¼ 0:0025 0:0625 0:1 0:5  : ð43Þ Let C¼ 0:01½ 0:01 T; ð44Þ H¼ 0:0025 0:1 0:0625 0:5  ; ð45Þ Q¼ R ¼ 1 0 0 1  ; ð46Þ W ¼ 1 ð47Þ and V ¼ 0:01 0 0 0:01  : ð48Þ

In the LEQG design, the Kalman gain must be solved firstly. The solution M for the ARE is determined to be

M ¼ 0:0102 0:0024 0:0024 0:0017



: ð49Þ

The Kalman filter gain can then be calculated as K¼ 0:0250 0:1741

0:0168 0:0955



: ð50Þ

4.1. Case of actuators are normal

First, we consider all actuators function normally with B¼ ½BX0 BX . For the

design of LEQG controller with the LEQG tunable scalar l¼ 0:02 in this paper and Kalman gain in (50), we obtain the solutions H and K for AREs as

H¼ 20:2428 0:0904 0:0904 2:5243  ð51Þ and K¼ 20:2455 0:0906 0:0906 2:5244  : ð52Þ

The corresponding optimal state feedback control gain for the standard design can hence be calculated as

C¼ 0:0391 0:1522 0:7668 0:7656



: ð53Þ

The eigenvalues of the uncontrolled version of system (1) are double roots of 1, which imply the instability of system (1). However, the eigenvalues of the controlled system are found to be 0.9502 and 0.6039. It is obvious that the closed-loop dynamics of system (1) is stabilized by LEQG control gain matrix C.

4.2. Case of actuators may be abnormal

Now, we study the reliable design for possible abnormal functioning of actuators. Following the design procedure in Section 3 with B and R are re-placed by BX0 and RX0, respectively, we obtain the reliable design solutions H

and K for AREs (36) and (26) as H¼ 28:8645 10:0494 10:0494 14:6868  ð54Þ and K¼ 28:8723 10:0535 10:0535 14:6890  : ð55Þ

The corresponding optimal state feedback control gain of the proposed reliable design is attained from (40) as

C¼ 0:0484 0:2886 1:2546 1:4925



: ð56Þ

The eigenvalues of the controlled system are found to be kðA BCÞ ¼ 0:9593 and 0.1870. The scalars in Theorem 2 are calculated to be a¼ 0:9317, b ¼ 0:1847 and c¼ 0:8937. From Theorem 2, the bounds of gain margins for nX0_i

and nXiare depicted in Fig. 2. From Fig. 2, A ðBN ÞKC is stable for nX0 ¼ 1

and 0:0544 < nX<2:3988.

Table 1 below shows the reliable controller design with N ¼ 200 in (18) and the initial value xx0¼ ½1 0 T.

From Table 1, the standard LEQG design makes the system to be unstable for BX¼ 0. However, the proposed reliable design still can tolerate the case for

BX¼ 0 with the tradeoff for higher performance index.

5. Conclusions

In this paper, we have studied the reliable stabilization of discrete-time systems using LEQG approach. A procedure has been derived for the design of reliable discrete LEQG control. The key is to find the ARE solutions H and K for reliable controller, which maintains system stability despite the abnor-mal operation of actuators. In additions, the bounds of gain margins for the feedback control is also obtained as in Theorem 2. In contrast to the traditional LQG method, LEQG control design explicitly takes both the system and measurement covariances into consideration. Moreover, the LQG control law was shown to be a special case of the LEQG control when tunable scalar l tends to zero.

Table 1

Comparisons of different control designs

System characteristic Performance index from (17) Closed-loop eigenvalues ½BX0B_X (standard design)

Normal system½BX0B_X 1.2407 0.9502, 0.6039

Faulty system with½BX00 Not available 1.0083, 0.9765

BX0only (reliable design)

Normal system½BX0B_X 1.2514 0.9593, 0.1870

Faulty system with½BX00 10.8352 0:9855 0:0057i

Appendix A. Development of LEQG control algorithm

Eqs. (17) and (18) can be rewritten in terms of a nested conditional expec-tation as PI¼ E EjD0 EjD1    EjDNl exp l 2U h i n o    n o n o n o : ðA:1Þ Let WðDkÞ ¼ EjDk exp l 2U h i n o : ðA:2Þ

A recursive formula for WðDkÞ can be obtained from (A.1) and (A.2) as

WðDkÞ ¼ EjDkfWðDkþ1Þg for k¼ 0; . . . ; N 1: ðA:3Þ Moreover, WðDNÞ ¼ EjDN exp l 2 ^xx T NQNxx^N " ( ( þX N 1 k¼0 ð^xxT_kQk^xxkþ uTkRkukÞ #)) ¼ aNexp l 2 xx^ T NHN^xxN " ( þX N 1 k¼0 ð^xxT_kQkxx^kþ uTkRkukÞ #) ; ðA:4Þ where aN  1 and HN  QN.

From Eqs. (9)–(11), (A.3) and (A.4), we then have

WðDN 1Þ ¼ EjDN 1 exp l 2 xx^ T NHN^xxN " ( ( þX N 1 k¼0 ð^xxT_kQkxx^kþ uTkRkukÞ #)) ¼ EjDN 1 exp l 2 ðxxN " ( ( þ KNsNÞ T HNðxxNþ KNsNÞ þX N 1 k¼0 ð^xxT_kQk^xxkþ uTkRkukÞ #)) ¼ exp l 2 XN 1 k¼0 ð^xxT_kQk^xxk ( þ uT kRkukÞ ) Z 1 1 exp l 2½ðxxN n þ KNsNÞ T

 HNðxxNþ KNsNÞ o ₁ ð2pÞnjSNj ½ 1=2 exp  1 2s T NS 1 N sN  dsN ¼ exp l 2 XN 1 k¼0 ð^xxT_kQkxx^k ( þ uT kRkukÞ ) Z 1 1 1 ½ð2pÞnjSNj 1=2  exp  1 2½s T NðSN 1 lK T NHNKNÞsN lxxNTHNxxN lðKNsNÞ T HNxxN lxxTNHNKNsN  dsN: ðA:5Þ Suppose S 1 N >lK T

NHNKN. It is clear that Eq. (A.5) can be rewritten as

WðDN 1Þ ¼ jðS 1 N lK T NHNKNÞ 1j jSNj " #1=2 exp l 2 XN 1 k¼0 ð^xxT kQk^xxk ( þ uT kRkukÞ )  exp  1 2½ lxx T NHNxxN ðlKNTHNxxNÞ T ðS 1 N lKNHNKNÞ 1  ðlKT NHNxxNÞ  Z 1 1 1 ½ð2pÞnjðS 1 N lKNTHNKNÞ 1 j 1=2 *  exp  1 2½sN ðS 1 N lK T NHNKNÞ 1ðlKNTHNxxNÞ T ðS 1 N lK T NHNKNÞ½sN ðSN 1 lK T NHNKNÞ 1 ðlKT NHNxxNÞ  dsN + : ðA:6Þ Since the integral term in the brackethi of (A.6) equals one, we then have

WðDN 1Þ ¼ aN jðS 1 N lK T NHNKNÞ 1 j jSNj " #1=2 exp l 2 XN 1 k¼0 ð^xxT_kQk^xxk ( þ uT kRkukÞ )  exp l 2xx T N½HN n þ lHNKNðS 1 lKNTHNKNÞ 1 K_NTHN xxN o : ðA:7Þ Define WðDN 1Þ  aN 1exp l 2 xx T NKNxxN " ( þX N 1 k¼0 ð^xxT_kQkxx^kþ uTkRkukÞ #) : ðA:8Þ

From (A.7) and (A.8), we have KN  HNþ lHNKNðSN 1 lK

T

and aN 1 aN jðS 1 N lKNTHNKNÞ 1j jSNj " #1=2 ¼ aNjI lKNTHNKNSNj 1=2: ðA:10Þ

From Eqs. (15) and (A.8) can be rewritten as WðDN 1Þ ¼ aN 1exp l 2 ðAN 1xx^N 1 " ( þ BN 1uN 1ÞTKNðAN 1xx^N 1 þ BN 1uN 1Þ þ XN 1 k¼0 ð^xxT_kQkxx^kþ uTkRkukÞ #) : ðA:11Þ Let PI min u0;...;uN 1 EfWðDN 1Þg ¼ min u0;...;uN 2 Efmin uN 1 WðDN 1Þg: ðA:12Þ

By solving the optimal condition of (A.12) from oWðDN 1Þ

ou N 1

¼ 0; ðA:13Þ

we have the following optimal control law:

u_{N 1}¼ ðRN 1þ BTN 1KNBN 1Þ 1BTN 1KNAN 1^xxN 1: ðA:14Þ

Substituting (A.14) into (A.11), we have

PIðDN 1Þ  aN 1exp l 2 ^xx T N 1HN 1xx^N 1 " ( þX N 2 k¼0 ð^xxT_kQkxx^kþ uTkRkukÞ #) ; ðA:15Þ where HN 1 ¼ QN 1þ ATN 1½KN KNBN 1ðRN 1þ BTN 1KNBN 1Þ 1BTN 1KN AN 1: ðA:16Þ Applying the procedure above recursively backward in time, we will get the feedback control input u _{for each stage.}

Appendix B Let X  R þ BT_KB X1 X2 XT 2 X3  ; ðB:1Þ

where X1¼ RX0þ BT X0KBX0, X₂¼ BT X0KBX and X3¼ RXþ BTXKBX. We have ðA BNC_CÞT KðA BNC_{CÞ K} ¼ Q þ H K þ AT_KB X0ðRX0þ BTX0KB_X0Þ 1BT X0KA AT_KBNC_{C C}T_NC_BT_{KAþ C}T_NC_BT_KBNC_C ¼ Q lHKðS 1 lKT_SKÞ 1_KT_{Hþ A}T_KB I 0   ½I 0 ðR  þ BT_KBÞ I 0   1 ½I 0 BT_{KA C}T_{ðR þ B}T_KBÞNC_C CT_NC_{ðR þ B}T_{KBÞC þ C}T_NC_{ðR þ B}T_KBÞNC_{C C}T_NC_RNC_C  QC _CT_{YC; ðB:2Þ} where QC _{ Q þ lHKðS} 1 _lKT_SKÞ 1_KT H P0; ðB:3Þ Y  Y1 Y2 YT 2 Y3  ðB:4Þ and Y1 NX0RX0NX0 ðI NX0ÞX1ðI NX0Þ; ðB:5Þ Y2 ðI NX0ÞX₂ðI N_XÞ; ðB:6Þ Y3 X3 X2TX 1 1 X2þ NXRXNX ðI NXÞX3ðI NXÞ: ðB:7Þ

It is known (e.g. Gajic and Qureshi [12]) that A BNC_{C is stable if}

ðA BNC_CÞT_{KðA BN}C_{CÞ K 6 0. Thus, from (B.2) the matrix A BN}C_Cis

stable if Y > 0.

By employing elementary row and column operations on matrix Y, we have I Y2Y3 1 0 I  Y1 Y2 YT 2 Y3  I 0 Y 1 3 Y2T I  ¼ Y1 Y2Y3 1Y2T 0 0 Y3  : ðB:8Þ

It is obvious from (B.8) that the matrix Y > 0 if Y3>0 and Y1 Y2Y3 1Y T 2 >0. Similarly, we have I 0 YT 2Y1 1 I  Y1 Y2 YT 2 Y3  I Y 1 1 Y2 0 I  ¼ Y1 0 0 Y3 Y2TY1 1Y2  : ðB:9Þ

This leads to the result of Y > 0 if Y1>0 and Y3 Y2TY1 1Y2>0. Now, we try

a¼ ½kminðRX0Þ=kmaxðX1Þ 1=2 ; ðB:10Þ b¼ kminðRXÞ=kmaxðX3Þ; ðB:11Þ c¼ kminðX3 X2TX1 1X2Þ=kmaxðX3Þ; ðB:12Þ d¼ kmaxfX2T½RX0NX20 X1ðI NX0Þ 2 1X2ðI NX0Þ 2 g=kmaxðX3Þ ðB:13Þ and f ¼ kmaxfX2½ðX3 X2TX 1 1 X2Þ þ RXNX2 X3ðI NXÞ2 1X2TðI NXÞ2g =kmaxðX1Þ: ðB:14Þ

Here kmaxðÞ and kminðÞ denote the maximum and minimum eigenvalues of a

matrix, respectively.

Case 1: (Conditions for Y1>0 and Y3 Y2TY1 1Y2>0)

First, we consider the condition for Y1>0. Since 0 < a < 1, from (B.5) we

then have Y1>0 if

kminðRX0ÞN2

X0 >kmaxðX1ÞðI NX0Þ2:

Using (B.10), we have Y1>0 if

½ð1 aÞNX0 I ½ð1 þ aÞNX0 I < 0:

This leads to the result of Y1>0 if

1 1þ aI < NX0 < 1 1 aI; i:e:; 1 1þ a< nX0i< 1 1 a for all i¼ 1; . . . ; r: Next, we solve for the condition Y3 Y2TY1 1Y2>0.

It is observed from (B.5)–(B.7) that Y3 Y2TY 1 1 Y2¼ RXNX2þ ðX3 X2TX 1 1 X2Þ X3ðI NX2Þ ðI NXÞ  XT 2ðI NX0Þ½RX0NX20 X1ðI NX0Þ 2 1ðI NX0ÞX2ðI NXÞ: We then have Y3 Y2TY1 1Y2>0 if kminðRXÞNX2þ kminðX3 X2TX 1 1 X2Þ kmaxðX3ÞðI NX2Þ kmaxfX2T½RX0N2

X0 X1ðI NX0Þ2 1X₂ðI N_X0Þ2gðI N_XÞ2>0:

Using the notation defined in (B.11)–(B.13), we have Y3 Y2TY1 1Y2>0 if

bN2 Xþ cI ðI NXÞ 2 dðI NXÞ 2 >0:

From (41), the sufficient condition for Y3 Y2TY1 1Y2>0 can be rewritten as bn2_Xiþ c ð1 nXiÞ 2 dð1 nXiÞ 2 >0 for all i¼ 1; . . . ; m r:

Case 2: (Conditions for Y3>0 and Y1 Y2Y3 1Y T 2 >0)

Similarly, since 0 < b < 1 and 0 < c < 1, we have Y3>0 if

kminðRXÞNX2þ kminðX3 X2TX 1

1 X2Þ > kmaxðX3ÞðI NXÞ 2

: That is, we have Y3>0 if

½ð1 bÞNX ð1 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi bþ c bc p ÞI ½ð1 bÞNX ð1 þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi bþ c bc p ÞI < 0: The sufficient condition for Y3>0 can also be rewritten as

1 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffibþ c bc 1 b I < NX< 1þpffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffibþ c bc 1 b I; i:e:; 1 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi bþ c bc p 1 b < nXi< 1þpffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffibþ c bc 1 b for all i¼ 1; . . . ; m r: From (B.5)–(B.7), Y1 Y2Y3 1Y T 2 ¼ RX0NX20 X1ðI NX0Þ 2 ðI NX0ÞX2ðI NXÞ  ½ðX3 X2TX1 1X2Þ þ RXNX2 X3ðI NXÞ 2 1ðI NXÞX2TðI NXÞ: Thus, we have Y1 Y2Y3 1Y T 2 >0 if kminðRX0ÞN2

X0 kmaxðX1ÞðI NX0Þ2 k_maxfX₂½ðX₃ XT

2X 1 1 X2Þ þ ðRXÞNX2 X3ðI NXÞ 2 1X₂TðI NXÞ 2 gðI NX0Þ 2 >0:

From (B.10) and (B.14), the above condition can be simplified as a2_N2

X0 ðI N_X0Þ2 f ðI N_X0Þ2>0:

i:e:; a2_n2

X0_i ð1 n_X0_iÞ2 f ð1 n_X0_iÞ2>0 for all i¼ 1; . . . ; r:

Results of Theorem 2 are hence implied.

Acknowledgements

This research was supported by the National Science Council, Taiwan, R.O.C. under Grants NSC 90-2212-E-009-067, NSC 90-2213-E-009-102, and by the Ministry of Education, Taiwan, R.O.C. under Grant EX-91-E-FA06-4-4.

References

[1] J.-M. Lin, S.-W. Lee, Bank-to-turn optimal guidance with linear exponential quadratic Gaussian performance criterion, AIAA J. Guidance, Control Dyn. 18 (1995) 951–958. [2] J.-M. Lin, System designed by linear exponential quadratic Gaussian and loop transfer

recovery methodology, AIAA J. Guidance Control Dyn. 18 (1995) 189–192.

[3] D.-C. Liaw, Y.-W. Liang, T.-C. Lee, Reliable control of nonlinear systems, IEEE Trans. Auto. Contr. 45 (4) (2000) 706–710.

[4] D.-C. Liaw, Y.-W. Liang, Quadratic polynomial solutions of the Hamilton–Jacobi inequality in reliable control design, IEICE Trans. Fundam. Electron., Commun. Comput. Sci. E81-A (9) (1998) 153–160.

[5] R.J. Veillette, Reliable linear-quadratic state-feedback control, Automatica 31 (1995) 137–143. [6] R.F. Stengel, Stochastic Optimal Control, John Wiley and Sons, New York, 1986.

[7] R.G. Brown, P.Y.C. Hwang, Introduction to Random Signals and Applied Kalman Filtering, John Wiley and Sons, New York, 1997.

[8] T. Kailath, Linear System, Prentice-Hall, Englewood Cliffs, NJ, 1980.

[9] G. Chen, G. Chen, S.H. Hsu, Linear Stochastic Control System, CRC Press, Boca Raton, FL, 1995.

[10] R.R. Bitmead, M. Gevers, V. Wertz, Adaptive Optimal Control, Prentice-Hall, Englewood Cliff, NJ, Australia Pty. Ltd., 1990 (Chapter 4).

[11] S. Bittanti, A.J. Laub, J.C. Willems, The Riccati Equation, Springer-Verlag, Berlin, Heidelberg, 1991 (Chapter 10).

[12] Z. Gajic, M.T.J. Qureshi, Lyapunov Matrix Equation in System Stability and Control, Academic Press, San Diego, CA, 1995 (Chapter 1).

[13] M.G. Safanov, Stability and Robustness of Multivariable Feedback Systems, MIT Press, Cambridge, MA, 1980 (Chapter 4).