Summary

A novel hard-decision CFA interpolation procedure has been developed based on the spectral-spatial correlation of a Bayer mosaic image. The proposed HPHD CFA interpolation method effectively reconstructs fine detail features in both edge and texture regions of demosaiced images. One merit of the proposed HPHD algorithm is that it can combine with many existing image interpolation methods such as decision-based algorithm (set α =1), edge-directed interpolation, adaptive interpolation, linear interpolation, etc for improved performance. Moreover, the proposed heterogeneity-projection scheme provides an efficient method for decision-based algorithms to make accurate direction-selection before performing interpolation.

In order to reconstruct the demosaiced images with fewer color artifacts, a novel CDEA CFA interpolation method is then proposed to combine with the HPHD algorithm. The proposed CDEA algorithm effectively reduces color artifacts in both smooth and edge regions of demosaiced images. Furthermore, any existing image interpolation method can be

combined with the proposed CDEA algorithm to reconstruct the green channel. A new edge-adaptive interpolation method is also presented by adopting the hard-decision rule from HPHD algorithm to reconstruct the green channel from CFA samples. In Chapter 5, the performance of the proposed HPHD-CDEA method will be compared with three renowned CFA interpolation methods. Experimental results will show that HPHD-CDEA method not only outperforms all of them in PSNR (dB) and S-CIELAB ∆E^*_ab measures, but also gives superior demosaiced fidelities in visual comparison.

Chapter 3

Robust Mobile Robot Visual Tracking Control Based on a Dual-Jacobian Visual Interaction Model

3.1 Introduction

From the literature discussed in Chapter 1, we have noted that a challenge in mobile robot visual tracking control design is to develop a visual tracking control system to track a dynamic moving target based on a stability criterion and overcome the uncertainties encountered in practical systems. This problem motivates us to derive a new model for designing a robust VTC to solve the visual tracking problem of dynamic moving target and overcome the internal disturbances of practical system (such as system parametric uncertainty and velocity quantization uncertainty). To achieve this, this chapter presents a novel dual-Jacobian visual interaction model to help the design of a robust VTC for a wheeled mobile robot equipped with a tilt camera. The proposed design enhances various image-tracking applications using an on-board monocular camera, such as human-robot interaction and surveillance. Based on Lyapunov theory, the proposed control scheme not only possesses some degree of robustness against parametric uncertainty, but also overcomes the external uncertainty caused by velocity quantization noise. Moreover, the proposed controller fully works in image space; hence the computational complexity and the effects of sensor/camera modeling errors can be greatly reduced. The main differences between the proposed VTC and other existent approaches are summarized as follows:

1) The proposed dual-Jacobian visual interaction model considers not only the effect of

mobile robot motion, but also the effect of target motion. Thus, based on the proposed model, the visual tracking control problem of a unicycle-modeled mobile robot for tracking a dynamic moving target can be solved with exponential convergence using a single controller. Moreover, the proposed model also considers the kinematics of a tilt camera platform mounted on the mobile robot. Therefore, the applicability of the proposed method is greatly increased.

2) The proposed visual tracking control system not only possesses some degree of robustness against the system model uncertainties, but also overcomes the unmodelled quantization effect in the velocity commands and the occlusion effect during visual tracking process.

This advantage enhances the reliability of the proposed method in practical applications.

3) The proposed visual tracking control system works fully in image space. Therefore, compared with position-based [23], homography-based [13, 14], and epipole-based [4, 15, 16, 17] visual tracking control approaches, the computational complexity and the sensor/camera modeling errors can be reduced due to the advantages of image-based visual servo control [2].

The basic assumptions of the proposed method are listed as follows:

1) The on-board camera is supposed to be a calibrated pinhole camera. Because the proposed VTC possesses some degree of robustness against parametric uncertainty, a simple linear camera calibration method [44] can be used to estimate the intrinsic parameters of the camera.

2) The width of target is supposed to be a priori known constant in order to simplify the depth estimation problem. However, this is not a necessary assumption for the proposed method.

Any algorithm or sensor which provides the depth information can be utilized to combine with the proposed method.

The rest of this chapter is organized as follows. Section 3.2 describes the system modeling of visual tracking control problem and the proposed dual-Jacobian visual interaction model accordingly. Section 3.3 presents the results of VTC design. In Section 3.4, the robustness of the control system against the system model uncertainty is analyzed and discussed. Section 3.5 presents the design of the robust control law to overcome the velocity quantization error encountered in practical systems. Section 3.6 summarizes the contributions of this design. Experimental results will be reported in Chapter 5. Several interesting experimental observations will be presented and discussed.

3.2 Camera-Object Visual Interaction Model

This section derives the visual interaction model between a mobile robot and a dynamic moving target. We first introduce the kinematics model of wheeled mobile robot and target used in this design. The mathematic derivations of the proposed model are then presented and explained.

3.2.1 Kinematics Model of Wheeled Mobile Robot and Target

Figure 3-1 shows the model of wheeled mobile robot and target considered in the nonholonomic visual tracking control problem. The wheeled mobile robot equips with a tilt camera to track a dynamic motion target, which is supposed to be a well-recognizable object with appropriate dimensions in the image plane and can only translate with respect to the robot. The tilt camera is mounted on top of the mobile robot and its optical-axis faces the target of interest, for instance, a human face. Figure 3-1(a) illustrates a model of the wheeled mobile robot and the target in the world coordinate frame Ff (see Fig. 3-2), in which the motion of the target is supposed to be holonomic such that

t f t

f

V

X

& = , (3.1)

(a) (b)

Fig. 3-1: (a) A model of the wheeled mobile robot and the target in the world coordinate frame. (b) Side view of the wheeled mobile robot with a tilt camera mounted on top of it.

where

X

^t_f =[

x

^t_f

y

^t_f

z

^t_f]^T and

V

_f^t =[

v

^x_f

v

^y_f

v

^z_f]^Tdenote, respectively, the position and velocity of target in the world coordinates.

Figure 3-1(b) is the side view of the scenario under consideration, in which the tilt angle φ gives the relationship between the camera coordinate frame

F and the mobile coordinate

_c frame

F . The kinematics of the wheeled mobile robot is described by [45]

_m

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

m f m f

m f m f m f

v v X

θ θ

cos 0 sin

& , and

m t

m f m f

w w

=

= φ θ

&

&

, (3.2)

where

X

^m_f =[

x

^m_f

y

^m_f

z

^m_f]^T is the position of mobile robot in the world coordinates, )

,

(θ^m_f φ are the orientation angle of mobile robot and the tilt angle of onboard camera, w_t^m is the tilt velocity of the camera, and (

v

^m_f,

w

^m_f) are the linear and angular velocities of mobile robot. In practice, (

v

^m_f,

w

^m_f ) can be used to calculate the velocity of each wheel of the mobile robot such that

2

where (v_l^m,

v ) are the left- and right-wheel velocities, respectively, and D represents the

^m_r distance between the two drive wheels. In the rest of this chapter, the target model (3.1) and mobile robot model (3.2) will be utilized to derive the visual interaction model and to design the visual tracking control system.

3.2.2 Coordinate Transformation from World Frame into Camera Frame

Figure 3-2 illustrates the relationship between coordinate systems, namely the world, camera and image coordinate frames. Let

X

_f =

X

^t_f −

X

^m_f denote the related position between mobile robot and the target in the world coordinate frame. In order to describe a mobile robot interacting with the target in the image coordinate frame, a visual interaction model has been derived by transferring the kinematics of X_f from the world coordinate frame into the image coordinate frame. This subsection presents the transformation of the kinematics of X_f from the world coordinate frame into the camera coordinate frame.

As shown in Fig. 3-2, X_c =[x_c y_c z_c]^T denotes the related position in the camera coordinate frame and can be calculated by the coordinate transformation such that

δY

is the distance between the center of robot head and the onboard camera. Because δY is a constant translation vector, the derivative of (3.4) becomes

Fig. 3-2: World, camera and image coordinate frames of robotic visual interaction.

Substituting (3.1), (3.2) and (3.4) into (3.5), the kinematics of the interaction between robot and target in camera frame can be obtained by taking the derivative of X_c such that

[ ]

⎥⎥ the following, the kinematics model (3.6) will be used to derive the interaction model in the image coordinate frame.

3.2.3 Coordinate Transformation from Camera Frame into Image Frame

In this subsection, the related position Xc is transformed into the image coordinate frame for deriving the visual interaction model based on (3.6). We first define the system state in the image coordinate frame for the controller design. Fig. 3-3 illustrates the definition of observed system state in the image plane. In Fig. 3-3,

x and

_i

y are, respectively, the horizontal and

_i vertical position of the centroid of target in the image plane, and

d is the width of target in

_x the image plane. Similar to the human’s visual tracking behavior, the purpose of the visual tracking control design is to control the centroid position and width of target from an initial state into the desired state in the image plane.

In the following, the visual interaction model is derived by (3.6) and the selected system state. Based on the pinhole camera model, the diffeomorphism (please see [46] for the specification) in the image plane can be defined by the standard projection equations [44]

such that:

Fig. 3-3: The definition of observed and desired system states in the image plane.

where (f_x,f_y) represent fixed focal length along the image x-axis and y-axis, respectively [47], and W denotes the actual width of the target. By taking the derivative of (3.7), the kinematic relationship between image and camera coordinate frames can be found such that

c

Substituting (3.6) and (3.7) into (3.8), the kinematic relationship between robot and target in the image coordinate frame can be modeled by quasi-linear parameter-varying (Quasi-LPV) description [48] such that

i

⎥⎥ elements of system matrix

A and vector

_i

C are time-varying function dependent on the

_i robot’s pose and target’s velocity; and the elements of control matrix

B are time-varying

_i function dependent on the robot’s pose and current system state.

3.2.4 Dual-Jacobian Visual Interaction Model

The visual interaction model (3.9) indicates that the elements of system matrix

A and

_i vector

C are function of target velocity. Thus, expression (3.9) can be rewritten such that

_i

u

Expression (3.10) shows that the visual interaction model consists of two parts: first, the effect of target motion

[ ]

i f^t

& . Thus, (3.10) can be rewritten as a dual-Jacobian equation such

Fig. 3-4: Depicts the concept of dual-Jacobian visual interaction model (3.11).

that

u V

X X

X

&_i = &_i^t+ &_i^m =

J

_{i f}^t +

B

_i , (3.11)

where matrix

J , which is named target image Jacobian, transfers the target velocity

_i

V

_f^t into target image velocity X&_i^t; matrix B_i, which denotes robot image Jacobian, transfers the mobile robot control velocity

u into robot image velocity

X&_i^m. In other words, the image velocity X&_i is caused by the combination of target image velocity X&_i^t and robot image

velocity X&_i^m. Figure 3-4 shows the concept of dual-Jacobian equation (3.11). Therefore, the visual interaction between robot and target in the image coordinate frame can be modeled as a dual-Jacobian visual interaction model (3.11), which combines the motion effect of mobile robot with moving target together.

Remark 3.1: The scalars k_x =f_x z_c and k_y =f_y z_c in (3.7) depend on the depth information between camera and target. The estimation of depth information is a demanding task in visual tracking control design; especially only one camera is used. Thus, an algorithm or sensor which provides the depth information is usually required during visual tracking

process. In order to simplify the depth estimation problem, an alternative is to assume that the width of target is known a priori. Therefore, the scalars

k and

_x k_y can be calculated using

the state variable

d directly based on the fact that

_x k_x =f_x z_c =d_x W and k_y =k_xf_y f_x, where W denotes the width of target for a specific target.

3.3 Visual Tracking Controller (VTC) Design

In this section, a visual tracking control law based on the proposed dual-Jacobian visual interaction model (3.11) for tracking a target of interest in the image plane is derived by exploiting feedback linearization and pole placement approaches.

3.3.1 Error Coordinate Transformation

In order to control the system from an initial state to the desired state, an error-state model will be helpful for us to design the tracking controller. Define the error state in the image plane such that

[ ] [

i i i i x x

]

^T

T e e e

e

x y d x y d

X

= = x − ^* y − ^* d − ^* , (3.12)

where Xi =

[

xi yi dx

]

^T is the vector of fixed desired states in the image plane;

[

i i x

]

^T

i x y d

X^* = ^{* * *} is the vector of estimated states from the VSE (see Chapter 4). Based on the error state (3.12), the dynamic error state model in the image plane can be derived directly by taking the derivative of (3.12) such that

u V X

X

X

&_e =− &_i^t− &_i^m =−

J

_{i f}^t −

B

_i . (3.13) With the new coordinate

X , the visual tracking control problem is transformed into a

_e stability problem. If

X converges to zero, then the visual tracking control problem is

_e solved.

3.3.2 Visual Feedback Control Design

Based on the dynamic error state model (3.13), we choose the feedback linearization control law such that

) where K_g is an 3-by-3 gain matrix. Substituting (3.14) into (3.13) yields

e g

e

X

X

& =−

K

. (3.15) Next, we choose the gain matrix such that

) Suppose that the initial error-state

X

_e(

t

₀) is within the image plane. Then expression (3.17) indicates that the following inequality:

) value of the vector X. From (3.19), it is clear that the system error-state satisfies

) control problem is solved. Summarizing the above discussions, we obtain the following theorem.

Theorem 3.1: Suppose the the initial system state Xi is within the image plane. Let

0

( _{1 2 3} > be three positive constants. Consider the closed-loop visual tracking system (3.13). If the matrix

B

_i is nonsingular, then the closed-loop visual tracking system (3.13) is exponentially stable by using the control law

)

Proof: Consider the closed-loop visual tracking system (3.13). We first define a

positive-definite Lyapunov function associated with the system error-state )

Taking the derivative of (3.21) yields

) that if f(u)>0 then the equilibrium point of (3.13) is asymptotically stable. Substituting the control law (3.20) into f(u), we then have

e symmetric positive definite (SPD) matrix, the following inequality holds:

0

where )λ_min(

A

denotes the minimum eigenvalue of matrix A. Expression (3.24) concludes that the closed-loop visual tracking system (3.13) is exponentially stable and hence completes

the proof. ■

Remark 3.2: Although the proposed image control law (3.20) results in a smooth convergence in the image plane, it still has to proof that the robot should have followed the target. The discussion of this problem is presented in Appendix D.

3.3.3 Singularity Analysis

The feedback linearization control law (3.20) poses a singularity problem of matrix

B

_i. Let

B denote an element of matrix

_mn

B

_i corresponding to the m-th row and n-th column.

By directly computing, the determinant of matrix

B

_i is given by

32 Based on (3.25), the singularity condition of matrix

B

_i can be found such that

φ following geometric relationship:

y

equals to 0 or π. The physical meaning of this is that the target is directly above or directly

Fig. 3-5: Physical meaning of the singularity condition (3.26).

below the robot, and the robot will be unable to approach the target in any way due to insufficient degrees-of-freedom. Therefore, the robot will stop tracking temporarily under such circumstances.

Remark 3.3: The proposed visual interaction model (3.11) poses a question that the derivation of d_x =k_xW is not always hold for a cylindrical target used in the system modeling. As shown in Fig. 3-6, parameters W, fx and zc remain the same, but the cylindrical target is shifted along xc. The projection dx is not the same and includes an error δ . Because the camera is d_x supposed to have a limited field of view, the error δ would be small and could be modeled d_x as a system uncertainty. In the next section, the robustness analysis is presented in order to handle this kind of uncertainty.

3.4 Robustness Against System Model Uncertainty

In this section, the robustness of the VTC (3.20) is investigated against model uncertainties from camera parameters (f_x,f_y) , robot parameters (θ^m_f ,φ) , and target parameters )(W,

x

&_i^t,

y

&_i^t,

d

&_x^t , etc. Consider the following closed-loop visual tracking system with parametric uncertainties:

Fig. 3-6: Projection error in d_x.

u V

u V

X

&_e =−

J

_{i f}^t −

B

_i =−(

J

_i+δ

J

_i) _f^t −(

B

_i +δ

B

_i) , (3.30) whereδ

J

_i and δ

B

_i are unknown bounded matrix-disturbances. Recall the positive-definite Lyapunov function defined in (3.21), the derivative of (3.21) with parametric uncertainties becomes

) (

| )]

( ) ( [ ] [

|

|_(3.30)

X X

_(3.30)

X V X u f u f u V

_(3.13)

f u

V

& = _e^T &_e =− _e^T

J

_{i f}^t + _e^T

B

_i =− +δ = & −δ , (3.31)

where δ

f

(

u

)=

X

_e^Tδ

J

_i

V

_f^t +

X

_e^Tδ

B

_i

u

is unknown. Assume that δ

J

_i and δ

B

_i are bounded and there exists two positive values

C and

_M

C such that

_N

e N e

M X C X

C u

f( )≤ ² +

δ . (3.32)

We now introduce the following definition.

Definition 3.1: The system (3.13) is said to be practically stable at the origin if a compact set

S in

ℜⁿ exists containing the origin such that for each

X

_e(

t

₀)∈

S

,

X

_e(

t

)∈

S

for all

t

0

t

≥ .

In practical applications, the practical stability problem can be more easily handled. Now, the

main result is presented as follows.

Theorem 3.2: Consider the closed-loop visual tracking system (3.13) with unknown bounded

parametric uncertainties δ

J

_i and δ

B

_i defined in (3.30). Let

C and

_M

C be two positive

_N values defined in (3.32) and assume that the target’s motion velocity V_f^t ≥0 is bounded and small enough. Choose the controller u as given in expression (3.20) with gain matrix

0

Then the origin is asymptotically stable under the condition V_f^t =0, and the origin is

practically stable under the condition V_f^t >0. ■ Proof: Choose the controller u in (3.20) with parametric uncertainties such that

)

maximum eigenvalue of matrix A, and A denotes the Euclidean norm value of the matrix

A. From (3.31), (3.33) and (3.34), it follows that

.

Expression (3.35) implies that if [λ_min(

K

_g)−

C

_M]

X

_e −

C

_N >0 can be guaranteed, then 0

|_(3.13)<

V&

is satisfied and thus the system has the robust property w.r.t. the parametric uncertainties.

From (3.34), the condition [λ_min(

K

_g)−

C

_M]

X

_e −

C

_N >0 can be rewritten such that

[

g i i

] [

g

]

N

N M g

e C C C

X >[λ_min(K )− ]⁻¹ = ρ_λ(K )− δB B^{−1 −1} λ_max(K )⁻¹ , (3.36) where ^{( )}

[

^{( )}

]

min^{( )}

1

max g g

g

K K

K

λ λ

ρ_λ = ⁻ . In general, we have that δB_iB_i⁻¹ <1 . By

assumption (i) of the theorem, expression (3.36) follows that when ρ_λ(K_g)→1 satisfies,

>0

∃ε such that

[ ] [ ]

{

^{ζ ∈ℜ ζ < ρλ − δ λ +ε}

}

→ ⁿ _{g i i}^{− −} _g ⁻ _N

e C

X : (K ) B B ^{1 1} _max(K ) ¹ . (3.37) Because of C_N = δJ_i −δB_iB_i⁻¹J_i V_f^t , the condition V_f^t =0 follows that

ε

e <

X

and

V

&^|(3.13)≤−

[

ρ_λ⁽

K

_g⁾− δ

B

_i

B

_i⁻¹

] X

_e ² <^0. (3.38) Thus, there exists a constant δ >0, δ <ε, ∃T such that

δ ε ⇒ <

< ( ) )

(

t

₀

X t

X

_{e e} ∀

t

≥

t

₀ +

T

, (3.39) which means that the origin is asymptotically stable under the condition V_f^t =0.

On the other hand, the condition V_f^t >0 follows that when ρ_λ(K_g)→1 satisfies,

>0

∃ε such that

0

|_(3.13)<

V&

for all X_e <

[

^ρλ(K_g)− ^δB_iB_i⁻¹

]

⁻¹

[

^λ_max(K_g)

]

⁻¹C_N +^ε . (3.40) Since V_f^t is supposed to be small enough, C_N ∝ V_f^t is sufficiently small. By assumption (ii) of the theorem, there exists a sufficiently large λ_max(K_g)and a constant δ₁ >0 such that

c

0

X

_e < , where c₀ =

[

^ρλ(K_g)− ^δB_iB_i⁻¹

]

^−1δ₁+^ε. Choose the set

S

=

{

ζ ∈ℜⁿ : ζ ≤

c

1

}

,

where 0

c

₁ > satisfies

c

₁ > . Then, any trajectory that begins at any

c

₀

X

_e(

t

₀)∈

S

will remain in S for all

t

≥ under the control law (3.20). Thus, the origin is practically stable

t

₀ under the condition V_f^t >0. This completes the proof. ■

In Chapter 5, the result of Theorem 3.2 will be validated by practical experiments.

Further, in realization of the control schemes, it was noted that the quantization error in velocity commands degrade the performance of the controller and might make the system unstable. It is therefore interesting to study the robustness issues related to velocity quantization uncertainty. In the following section, a robust control law based on Lyapunov’s direct method will be derived to overcome the velocity quantization uncertainty in practical control systems.

3.5 Robustness Against Velocity Quantization Error

When tracking a target, it is desirable for the robot to have a smooth motion in human-robot interaction. But in such circumstances, one will face the problem caused by velocity quantization error in practical implementation. In this section, a robust control law is derived to eliminate the velocity quantization error encountered in practical control systems based on the dynamic error state model defined in (3.13). To do so, a stability necessary condition (SNC) is first derived for ensuring asymptotic stability and practical stability of the closed-loop visual tracking system through Lyapunov’s direct method. The robust control law is then proposed to guarantee that the visual tracking system satisfies the SNC and hence complete the controller design.

3.5.1 Stability necessary condition (SNC)

Digital control systems usually have uniform quantization errors due to the finite-length

effects on the sample values [49]. In other words, the ideal (theoretical) control command u is quantized such that

u u

u = +δ , (3.41) where u denotes the practical (actual) control command sent to the robot actuator, and δ u represents the uniform quantization error encountered in the system. Thus, in practice, (3.22) becomes

0 ) (

| )]

( ) ( ) ( [ ) (

| _(3.13)

) 41 . 3 (3.13)

( =−

f u

=−

f u

−

f u

+

f u

=

V

−

f u

<

V

& δ δ δ & δ δ , (3.42)

where δf(δu)= X_e^TB_iδu−X_e^TδB_iδu≅ X_e^TB_iδu . Expression (3.42) shows that if the controller u satisfies the assumptions (i) and (ii) of Theorem 3.2, then the equilibrium point of the system (3.13) still can be unstable under the condition δ

f

(δ

u

)<0. Therefore, one has the following SNC in practical control implementation:

SNC: Consider the closed-loop visual tracking system (3.13) with unknown bounded parametric uncertainties δ

J

_i and δ

B

_i defined in (3.30). Let δ denote the velocity output u quantization error in practical systems. Suppose that the controller u given in (3.20) satisfies the assumptions (i) and (ii) of Theorem 3.2. Then, the result of Theorem 3.2 holds under the

condition δf(δu)= X_e^TB_iδu ≥0 ■

3.5.2 Proposed robust control law

SNC implies that a practical system may become unstable under the condition 0

) (

u

<

f

δ

δ . Our goal is to design a robust control law which not only guarantees SNC to be always satisfied but also increases the convergence rate of the control system.

First, we expand δ

f

( uδ ) such that

First, we expand δ

f

( uδ ) such that

在文檔中 影像感測器之色彩濾波陣列補插與輪式機器人之視覺追蹤控制設計 (頁 55-0)