FAT-based Adaptive Visual Servoing for Robots with Varying Uncertainties

Abstract—Most present adaptive control strategies for visual

servoing of robots have assumed that the unknown camera parameters, kinematics and dynamics of visual servoing system should be linearly parameterized in the regressor matrix form.

This is because the limitation of the traditional adaptive design in which the uncertainties should be time-invariant such that all time varying terms in the visual servoing system are collected inside the regressor matrix. However, derivation of the regressor matrix is tedious. In this paper, a FAT (function approximation technique) based adaptive controller is designed for visual servo robots without the need for the regressor matrix. A Lyapunov-like analysis is used to justify the closed-loop stability and boundedness of internal signals. Moreover, the upper bounds of tracking errors in the transient state are also derived. Computer simulation results are presented to demonstrate the usefulness of the proposed scheme.

I. I

NTRODUCTION

variety of approaches have been developed for the problem of visual servo control for robots over the years [1]-[3]. Most of them have assumed that the exact knowledge of calibration parameters of the camera and kinematics/dynamics of the robot are available. However, these parameters can never be precisely known in practical applications. Under this circumstance, one of the effective ways to deal with this difficulty is to apply the adaptive strategy to the visual servo control [4]-[12].

Bishop and Spong [13] developed an inverse dynamics type, position tracking control scheme with an online adaptive camera calibration control loop that guaranteed global asymptotic position tracking; however the exact model knowledge of the mechanical dynamic was required.

Zergeroglu et al. [14] designed an adaptive camera calibration controller using the backstepping procedure to produce globally uniformly ultimately boundedness in tracking errors with consideration of dynamic uncertainties. To eliminate the need for overparameterization, Astolfi et al. [15] used the adaptive immersion and invariance technique in the servoing loop.

However, this approach ignored the robot’s velocity-level dynamics by modeling the robot kinematics using simple

Manuscript received September 14, 2008. This work was supported in part by the Ministry of Economic Affairs of Taiwan, R.O.C., under Grant 8301XS3110.

M. C. Chien is with the Mechanical and Systems Research Laboratories, Industrial Technology Research Institute, No. 195, Sec. 4, Chung-Hsing Rd., Chutung, Hsinchu, 310, Taiwan, R.O.C. (corresponding author, phone:

886-3-591-8630;fax:886-3-591-3607;e-mail: [email protected]).

A. C. Huang is with the Department of Mechanical Engineering, National Taiwan University of Science and Technology. No. 43, Keelung Rd., Sec. 4, Taipei, Taiwan, ROC. (E-mail: [email protected]).

integrators. Akella [16] presented an adaptive controller to stabilize the complete robot-camera system by estimating only the nonlinearly parameterized camera orientation angle along with the linearly appearing mechanical/inertia parameters within the robot dynamics. Liu et al. [17] used the depth-independent interaction matrix making the unknown camera parameters appear linearly in the closed-loop dynamics so that a new adaptive algorithm is developed to estimate their values on-line. Extending the work in [17]-[20] to cope with robot dynamic uncertainties, Wang and Liu [21] employed the depth-independent image Jacobian which does not depend on the scale factors determined by the depths of feature points.

Extending the capability of the standard adaptive algorithm [22]

to deal with kinematics uncertainties, Cheah et al. [23]

proposed a novel dynamics regressor matrix using the estimated kinematics parameters. Hsu et al. [24] presented a Lyapunov-based design of model-reference adaptive control for multiple-input multiple-output systems with uniform relative degree two.

In the above development, calibration parameters of the camera and kinematics/dynamics of the robot have to be represented as a linear parametric form so that an adaptive controller can be designed. This is because the limitation of the traditional adaptive design in which the uncertainties should be time-invariant such that all time varying terms in the visual servoing system are collected inside the regressor matrix.

However, derivation of the regressor matrix for the visual servoing system is tedious. Besides, in the real-time realization, the regressor matrix has to be computed in every control cycle, and its complexity results in a considerable burden to the control computer. To ease the controller design and implementation, an adaptive controller is designed for the visual servoing system without the need for the regressor matrix in this paper. The unknown camera parameters, kinematics and dynamics of visual servoing system can be considered as time varying uncertainties and does not need to be linearly parameterized. The function approximation technique (FAT) [25]-[36] is employed to deal with these uncertainties, which plays an important role in the construction of the update laws.

The close loop stability and boundedness of internal signals are proved by using the Lyapunov-like technique with consideration of the approximation error. In addition, the upper bounds of tracking errors in the transient state are also derived.

Simulation results are presented to demonstrate the usefulness of the proposed scheme.

This paper is organized as follows. Section II formulates the robot dynamic equations and kinematics. In section III, we present our main results. Section IV presents simulation results

FAT-based Adaptive Visual Servoing for Robots with Time Varying Uncertainties

Ming-Chih Chien and An-Chyau Huang , Member, IEEE

A

Kobe, Japan, May 12-17, 2009

to illustrating the performance of the controller. Section V concludes the paper.

II. R

OBOT

D

YNAMICS AND

K

INEMATICS

In this section, a visual servo system for a robot manipulator is considered. We assume the camera is fixed above the robot workspace and can capture images throughout the entire robot workspace. The control objective is to move the robot end-effector to follow a desired trajectory in the Cartesian space via the camera observation of the end-effector position in the image space. In the following, the dynamics and kinematics of the visual servo system are introduced.

A. Dynamics

The dynamics of n-link rigid robot manipulator without considering friction or other disturbances can be described by D ( q ) q && + C ( q , q & ) q & + g ( q ) = τ (1) where q ∈ ℜ

ⁿ

is a vector of joint angles, D (q ) is the n × n inertia matrix, C ( q , q &) q & is the n-vector of centrifugal and Coriolis forces, g(q) is the gravitational force vector, and τ is the control torque vector. Although (1) presents a highly nonlinear and coupled dynamics, several good properties can be summarized as [22]

Property 1: D ( q ) = D

^T

( q ) > 0 and there exist positive constants α

1

and α

2

, α

₁

≤ α

₂

such that α

₁

I

_n

≤ D ( q ) ≤ α

₂

I

_n

for all q ∈ ℜ

ⁿ

.

Property 2: D & ( q ) − 2 C ( q , q & ) is skew-symmetric.

B. Kinematics

The position vector of the end-effector in the Cartesian space can be described as [23]:

) (q

x

_e

= Ω (2) where Ω ) ( ⋅ ∈ ℜ

ⁿ

→ ℜ

^m

is a non-linear transformation describing the relation between the joint space and task space.

The velocity vector of the end-effector in the Cartesian space is related to the joint-space velocity q& as:

q q J

x &

_e

=

_e

( ) & (3) where J

_e

( q ) = ∂ Ω ( q ) ∂ q ∈ ℜ

^m×n

is the Jacobian matrix which is assumed to be nonsingular in the working space. For the visual servo system, cameras are used to monitor the position of the end-effector in the image space. We use the standard pinhole camera model, which has been proven to be effective for most visual serving tasks [1]. The end-effector velocity x&

_e

can be mapped into the image space by the following relationship [1]:

x &

_c

= J

_c

( x

_e

) x &

_e

(4) where x&

_c

∈ ℜ

^k

is the time derivative of the image feature parameter vector x , and

_c

J

_c

(x

_e

) ∈ ℜ

^k×m

is the image Jacobian matrix. The image Jacobian was first introduced by Weiss et al. [37], who referred to it as the feature sensitivity

matrix. It is also refereed to as the interaction matrix [38] and the B matrix [39]. Using (3) and (4), x& is related to the

_c

joint-space velocity q& as [23]

x &

_c

= J

_i

( q ) q & (5) where J

_i

(q ) ∈ ℜ

^k×n

is the Jacobian matrix mapping from the joint space to the image space and J

_i

( q ) = J

_c

( x

_e

) J

_e

( q ) .

III. M

AIN

R

ESULTS

In this section, a FAT-based adaptive visual servo controller is proposed. We would like to consider the case when the precise forms of D (q ) , ), C ( q , q & g(q) in (1), and J

_i

(q ) in (5) are not available and their variation bounds are not given. This implies that traditional robust control is not applicable.

Moreover, it is well-known that derivation of the regressor matrix for the traditional adaptive control of rigid robot without visual servo is generally tedious. When including the visual servo into the adaptive control loop, the computation of the regressor matrix becomes extremely difficult. One of the contributions of the present paper is to propose a regressor-free adaptive controller for the visual servo robot system.

Define J

_i⁺

= J

^T_i

( J

_i

J

^T_i

)

⁻¹

∈ ℜ

^n×k

to be the pseudo inverse of )

(q

J

_i

, D

_j

= [ DJ

_i⁺

D J &

_i⁺

I

_n×n

] ∈ ℜ

^n×(2k+n)

as the augmented inertia matrix, C

_j

= CJ

_i⁺

∈ ℜ

^n×k

the augmented centrifugal and Coriolis matrix, and X = [ x &&

d

+ Λx

d

x &

r

Λ q & ]

^T

∈ ℜ

^2k+ⁿ

the augmented state vector. Define the signal vectors

r c

x

x x

s = & − & , x &

_r

= & x

_d

− Λe

_x

, s = q & − q &

_r

, and q &

_r

= J

_i⁺

x &

_r

, where x

_d

∈ ℜ

^k

is the vector of desired trajectories in the image space, e

_x

= x

_c

− x

_d

is the image tracking error, and

) ,..., ,

(

_{1 2 k}

diag λ λ λ

=

Λ with λ

_i

> 0 for all i=1, … k, is a matrix of gains. Then we can rewrite (1) as

τ g x C X D Cs s

D & + +

_j

+

_j

&

_r

+ = (6) A. Controller Design for Known Robot

Suppose D , C , g, and J are known, and we may design a

_i

proper control law such that τ follows the trajectory below

s K e K J g x C X D

τ =

_j

+

_j

&

_r

+ −

^T_{i p x}

−

_d

(7) where K and

_p

K are positive definite matrices. Substituting

_d

(7) into (6), the closed loop dynamics becomes

D s & + Cs + K

_d

s + J

^T_i

K

_p

e

_x

= 0 (8)

Define a Lyapunov function candidate as )

2

(

1 T p x

x

V = s

T

Ds + e K e Its time derivative along the trajectory of the closed loop dynamics can be computed as V & = − s

^T

K

_d

s + s

^T

( D & − 2 C ) s − s

^T

J

^T_i

K

_p

e

_x

+ e

^T_x

K

_p

e &

_x

(9) Since J

_i

s = J

_i

q & − x

_r

= s

_x

, and s

_x

= x &

_c

− x &

_r

= e &

_x

+ Λe

_x

, the above equation becomes

≤ 0

−

−

=

^T _d ^T_{x p x}

V & s K s Λe K e (10)

It is easy to prove that s and e are uniformly bounded and

_x

square integrable. Moreover, s& and e& are also uniformly

x

bounded. Hence, s → as 0 t → ∞ and e

_x

→ 0 as t → ∞ , or we may say x → x

_d

as t → ∞ and x & → x &

_d

as t → ∞ .

In summary, if all parameters in the visual servo system (1) and (5) are available, the torque controller (7) can give asymptotic convergence image tracking performance.

B. Controller Design for Uncertain Robot

When Jacobian matrix J is uncertain, we would like to

_i

define the estimation of x as

_c

q q J

x & ˆ

_c

= ˆ

_i

( ) & (11) where Jˆ is an estimation of

_i

J . Moreover, let us define the

_i

signal vectors s

_x

= ˆ x & −

_c

x &

_r

, x &

_r

= & x

_d

− Λe

_x

, s = q & ˆ − q &

_r

, and

r i

r

J x

q & ˆ = ˆ

⁺

& , where J ˆ

⁺_i

= J ˆ

^T_i

( J ˆ

_i

J ˆ

^T_i

)

⁻¹

is the pseudo inverse of Jˆ

i

. Now consider the case when D, C and g in (1) are not available, and controller (7) cannot be realized. A controller is able to be constructed based on (7) as

s K e K J g x C X D

τ = ˆ

_j

+ ˆ

_{j r}

+ ˆ − ˆ

^T_{i p x}

−

_d

(12) where Dˆ

_j

, Cˆ and gˆ are estimates of D

_j j

, C

j

, and g, respectively. Using (12), we may have the closed loop dynamics

g x C X D e K J s K Cs s

D & + +

_d

+ ˆ

^T_{i p x}

= − ~

_j

− ~

_j

&

_r

− ~ (13) where D ~

_j

D

_j

D ˆ

_j

−

= , C ~

_j

C

_j

C ˆ

_j

−

= and g ~ = g − g ˆ . If a proper controller and update laws for Dˆ ,

_j

Cˆ

_j

, gˆ , and Jˆ

_i

can be designed, we may have D ˆ

_j

→ D

_j

, C ˆ

_j

→ C

_j

, g ˆ → g , and

i

i

J

J ˆ → so that (13) can give desired performance. Since D

j

, C

j

, g, and J are functions of states and hence functions of time,

_i

traditional adaptive controllers are not applicable to give proper update laws except that the linearly parameterization assumption is feasible. On the other hand, since their variation bounds are not given, conventional robust designs do not work either. Here, we would like to use FAT to represent D

j

, C

j

, g, and J with the assumption that proper numbers of basis

_i

functions are employed

,

, ,

f J J g

g g

C C C D

D D

ε Z W J ε z W g

ε Z W C ε Z W D

+

= +

=

+

= +

=

i T T

T j T

j

(14)

where W

_D

∈ ℜ

^n2βD×n

, W

_C

∈ ℜ

^n2βC×n

, W

_g

∈ ℜ

^nβg×n

, and

k k _J×

ℜ

∈

^2β

W

J

are weighting matrices, Z

_D

∈ ℜ

^{n2βD×(2k+n)}

,

2 k

,

n _C×

ℜ

∈

^β

Z

C

z

_g

∈ ℜ

^nβg×1

, and Z

_J

∈ ℜ

^k2βJ×n

are matrices of basis functions, and ε are approximation error matrices.

_(⋅)

The number β

_(⋅)

represents the number of basis functions used.

Using respectively the same set of basis functions, the corresponding estimates can also be represented as

ˆ ˆ ˆ , , ˆ ˆ , ˆ ˆ

ˆ W

D

Z

D

C W

C

Z

C

g W

g

z

g

J W

J

Z

J

D

_j

=

^T _j

=

^T

=

^T _i

=

^T

(15)

Define ~

_{() ()}

ˆ

₍₎

⋅

⋅

⋅

= W − W

W , then equation (13) becomes

1

~

~ ~

ˆ

ε z W x Z W X Z W

e K J s K Cs s D

g g C C D

D

− − +

−

= +

+ +

r T T T

x T p i d

&

&

(16) where ε

₁

= ε

₁

( ε

_D

, ε

_C

, ε

_g

, s ) is a lumped approximation error.

Since W

_(⋅)

are constant matrices, their update laws can be easily found by proper selection of the Lyapunov-like function.

Let us consider a candidate

~ )}

~

~

~

~

~

~ ( ~

) {(

~ )

~ ,

~ ,

~ , , ,

(

₂¹

J J J g g g C C C D D D

J g C D

W Q W W Q W W Q W W Q W

e K e Ds s W W W W e s

T T

T T

x T p T x x

Tr V

+ +

+ +

+

= (17)

The matrices Q

_D

∈ ℜ

^{n2β ×D n2βD}

, Q

_C

∈ ℜ

^{n2β ×C n2βC}

, ,

g g n nβ × β

ℜ

g

∈

Q and Q

_J

∈ ℜ

^k2βJ×k2βJ

are positive definite weighting matrices. The time derivative of V along the trajectory of (16) can be computed as

ˆ ] ) ~

( ˆ

~

ˆ )

~ ( ˆ )

~ ( [

ˆ

₁

J J J g g g g

C C C

D D D D

D

W Q W W Q s z W

W Q s x Z W W Q Xs Z W

ε s e K e e K J s s K s

&

&

&

&

&

&

&

T T

T

T r T T

T

x T T p x x T p T i T d

Tr V

+ +

+

+ +

+

−

+ +

−

−

=

(18)

From (5), (8), (14), and (15), since x & ˆ

_c

= x &

_c

− ( J

_i

− J ˆ

_i

) q & , we have

e ˆ &

_x

= e &

_x

− W ~

_J^T

Z

_J

q & + ε

₂

(19) where e & ˆ

_x

= & x ˆ

_c

− x

_d

is an estimate of e and

_x

ε

₂

= ε

₂

( ε

_J

, e

_x

) is a lumped approximation error. According to (19), we can rewrite (18) as

ˆ )]

~ ( ˆ )

~ (

ˆ )

~ ( ˆ )

~ ( [

2 1

J J J

J g g g g

C C C

D D D D

D

W Q K e q Z W W Q s z W

W Q s x Z W W Q Xs Z W

ε K e ε s e ΛK e s K s

&

&

&

&

&

&

&

+ +

+ +

+ +

+

−

+ +

−

−

=

Tp Tx T T

T

rT T T

T

Tp Tx x T

T p x T d

Tr V

(20)

with J ˆ

_i

s = J ˆ

_i

q & − x

_r

= s

_x

, and s

_x

= x & ˆ

_c

− x &

_r

= e & ˆ

_x

+ Λe

_x

. By selecting the update laws as

ˆ ) ˆ (

ˆ ) ˆ (

ˆ ) ˆ (

ˆ ) ˆ (

1 1 1 1

J J J

J J

g g g g g

C C C

C C

D D D

D D

W K

e q Z Q W

W s

z Q W

W s

x Z Q W

W Xs

Z Q W

σ σ

σ σ

+

−

=

+

−

=

+

−

=

+

−

=

−

−

−

−

Tp Tx T

r T T

&

&

&

&

&

&

(21)

where σ

_(.)

are positive numbers, equation (20) becomes

ˆ ) ( ~ ˆ )

( ~

ˆ ) ( ~ ˆ )

( ~

J J J g g g

C C C D D D

W W W

W

W W W

W ε

E QE E

T T

T T

T T

Tr Tr

Tr Tr

V

σ σ

σ σ

+ +

+ +

+

−

& =

(22)

Where E = [ s e

_x

]

^T

, ε = [ ε

1

K

^Tp 2

ε ]

^T

, and ⎥

⎦

⎢ ⎤

⎣

= ⎡

p d

ΛK 0

0 Q K

is a positive definite matrix. Owing to the existence of ε and

₁

ε in (22), the definiteness of

2

V& cannot be determined. In the following, we would like to investigate closed loop stability in the presence of these approximation errors. It is very easy to prove the inequalities hold

⎟⎟ ⎠

⎜⎜ ⎞

⎝

⎛ −

−

≤ +

−

²

min 2 2 min

1

) ( ) 1

( ε

E Q Q ε

E QE

E

^{T T}

λ λ (23a)

~ ) ( ~ ) (

ˆ )

( W ~

₍^T_⋅)

W

_(⋅)

≤

₂¹

Tr W

₍^T_⋅)

W

_(⋅)

−

₂¹

Tr W

₍^T_⋅)

W

_(⋅)

Tr (23b)

Together with the relationship

~ )]

( ~ ) (

~ ) ( ~ ) (

~ ) ( ~ ) (

~ ) ( ~ ) ( )

( [

~ )]

~

~ ~

~

~

~ ( ~ [

max

max max

max 2 2 max

1 12

J J J

g g g C

C C

D D D

J J J g g g

C C C D D D

W W Q

W W Q W

W Q

W W Q E

A

W Q W W Q W

W Q W W Q W AE E

T

T T

T T T

T T

T

Tr

Tr Tr

Tr Tr

V

λ

λ λ

λ λ

+

+ +

+

≤

+ +

+ +

=

(24)

where ⎥

⎦

⎢ ⎤

⎣

= ⎡ K

p

0 0

A D , we may rewrite (22) into the form

[ ]

)]}

( ) ( [

)]

( ) (

[

~ ) ( ~ ] ) ( [

~ ) ( ~ ] ) ( [

~ ) ( ~ ] ) ( [

~ ) ( ~ ] ) ( [

) ( ) 1

( ) ( {

max max max max

2 min 2 min 2 max

1

J J J g g g

C C C D D D

J J J J

g g g g

C C C C

D D D D

W W W

W

W W W

W

W W Q

W W Q

W W Q

W W Q

Q ε E

Q A

T T

T T

T T T T

Tr Tr

Tr Tr

Tr Tr Tr Tr V

V

σ σ

σ σ

σ αλ

σ αλ

σ αλ

σ αλ

λ λ αλ

α

+ +

+ +

− +

− +

− +

− +

+

− +

−

& ≤

(25)

Although D is unknown, we know that ∃ D and D s.t.

D

D ≤ D ≤ , i.e. ∃ η

_A

, η

_A

> 0 s.t. λ

_max

( A ) ≤ η

_A

and η

A

λ

_min

( A ) ≥ [40]. We can select α to be

⎪⎭

⎪ ⎬

⎫

⎩ ⎨

≤ ⎧

) , (

) (

) , , (

) , (

) min (

max max

max max

min

J J g

g C C D

D

Q Q

Q Q

Q

λ σ λ

σ λ

σ λ

σ η

α λ

A

(26)

Then (25) becomes

)]

( ) ( ) (

) (

2 [ 1 ) ( 2

1

2

min

J J J g g g C C C

D D D

W W W

W W

W

W W Q ε

T T

T

T

Tr Tr

Tr

Tr V

V

σ σ

σ

λ σ α

+ +

+

+ +

−

& ≤

(27)

This implies V& < 0 whenever

)]

( ) ( ) (

) (

2 [ sup 1 ) ( 2

1

2

min 0

J J J g g g C C C

D D D

W W W

W W

W

W W Q ε

T T

T

T t

Tr Tr

Tr

Tr V

σ σ

σ

α σ

αλ

τ

+ +

+

+

>

≥

(28)

Hence, we have proved that s, e

x

, W ~

_D

, W ~

_C

, W ~

_g

and W ~

_J

are uniformly ultimately bounded. For further development, we may apply the comparison lemma [40] to (27) to have the upper bound for V as

)]

( ) (

) (

) (

2 [ 1

) sup ( 2 ) 1 ( )

(

²

min 0

) (

0 0

J J J g g g

C C C D D D

W W W

W

W W W

W

Q ε

T T

T T

t t t

t

Tr Tr

Tr Tr

t V e

t V

σ σ

σ α σ

αλ

τ

α

+ +

+ +

+

≤

< <

−

−

(29)

Using the inequality

~ )]

( ~ ) (

~ ) ( ~ ) (

~ ) ( ~ ) (

~ ) ( ~ ) ( [ ) (

min

min min

2 min 2 1 2 min

1

J J J

g g g C

C C

D D D

W W Q

W W Q W

W Q

W W Q E

A

T

T T

T

Tr

Tr Tr

Tr V

λ

λ λ

λ λ

+

+ +

+

≥

(30)

we may find the upper bound for E as

²

)]

( ) (

) (

) (

1 [

) sup ( ) 1

2 (

2

min 0

) 2 (

0 0

J J J g g g

C C C D D D

W W W

W

W W W

W

Q ε E

T T

T T

A

t A t

t t

A

Tr Tr

Tr Tr

t V e

σ σ

σ η σ

α

λ η α

η

τ

α

+ +

+ +

+

≤

< <

−

−

(31)

Therefore, we may compute the error bound as

2 1 min )

2( 0

)]

( ) (

) ( ) (

1 [

) sup ( 1 )

( 2

0 0

J J J g g g

C C C D D D

W W W

W

W W W

W

Q ε E

T T

T T

A

t A t

t t

A

Tr Tr

Tr Tr

t e V

σ σ

σ η σ

α

λ η

η α

τ

α

+ +

+ +

+

≤

<

<

−

−

(32)

This proved that the time history of the error signal is bounded by a weighted exponential function shifted with some constant.

The transient performance analysis is thus completed.

Theorem 1: Consider the servo system (1) and (5) with unknown parameters D, C, g and J

i

, then control input (12) and update law (21) ensure that

(i) error signals s, e

_x,

W ~

_D

,

W ~

_C

,

W ~

_g

, and W ~

_J

are u.u.b.

(ii) the bound of the tracking error vectors for t ≥ can be t

₀

derived as the form of (32), if the Lyapunov-like function candidates are chosen as (17).

Remark 1: The term with σ

_(⋅)

in (21) is to modify the update

law to robustify the closed-loop system for the effect of the

approximation error [27]. Suppose a sufficient number of basis

functions β

_(⋅)

is selected so that the approximation error can be

neglected then we may have σ

_(⋅)

= 0 , and (22) becomes

≤ 0

−

= E

^T

QE

V& (33) It is easy to prove that s and e are also square integrable.

_x

Moreover, s& and e& are also uniformly bounded. As a result,

x

asymptotic convergence of s and e

x

can easily be shown by Barbalat’s lemma. Hence, s → as 0 t → ∞ and e

_x

→ 0 as

∞

→

t , this further implies that x → x

_d

as t → ∞ and x

d

x & → & as t → ∞ even though D, C, g, and J

_i

are all unknown.

IV. S

IMULATION

S

TUDY

In this section, simulation results are presented to demonstrate the usefulness of the proposed controller. Consider a 2-DOF planar robot (Fig.1) represented by (1) and (5). The quantities m

i

, l

i

, l

ci

and I

i

are mass, length, gravity center distance and inertia of link i, respectively. Actual values of link parameters in the simulation [28] are m

1

=0.5kg, m

2

=0.5kg, l

1

=l

2

=0.75m, l

c1

=l

c2

=0.375m, I

1

=0.09375kg-m

²

, and I

2

=0.046975kg-m

²

. The Jacobian matrix J

_i

(q ) mapping from the joint space to the image space is given by [23]

⎥ ⎦

⎢ ⎤

⎣

⎡ +

−

−

⎥ −

⎦

⎢ ⎤

⎣

⎡

= −

12 2 12 2 1 1

12 2 12 2 1 1 2 1

0 ) 0

( l c l c l c

s l s l s l f

z f

c c

i c

β

q β

J (34)

where s

₁

= sin( q

₁

) , ) s

₁₂

= sin( q

₁

+ q

₂

, c

₁

= cos( q

₁

) , )

cos(

_{1 2}

12

q q

c = + , f

_c

is the focal length of the camera which is set to be 50mm, and z

_c

is the perpendicular distance between the robot and the camera which is set as 0.55m. The parameters

β

1

and β

₂

denote the scaling factors in pixels/m and 10000

2 1

= β =

β . We would like the end-point to track a 0.2m-radius circle centered at (0.8 m, 1.0 m) in 10 seconds without knowing its precise model. The initial conditions of the link angles and the motor angles are

[ 0.0022 1.5019 0 0 ]

^T

=

q (rad). The initial condition of

the image feature parameters vector is x

_c

= [ 800 750 ] . The controller gains are selected as

), 30 , 1 . 0 (

d

= diag

K Λ = diag ( 0 . 1 , 0 . 1 ), and

).

002 . 0 , 01 . 0 (

p

= diag

K Each element of D

j

, C

j

, g, and J

_i

is approximated by the first 81 terms of the Fourier series. The simulation results are shown in Fig. 2 to 4. Fig. 2 shows the tracking performance of the end-point and the desired trajectory in the image space (pixels). It is observed that the end-point trajectory converges nicely to the desired trajectory, although the initial position error is quite large. Fig. 3 is the tracking error (pixels). It shows that the transient response vanishes very quickly. Fig. 4 is the control inputs in torque (N.m). It is worth to note that in designing the controller we do not need much knowledge for the system. All we have to do is to pick some controller parameters and some initial weighting matrices.

V. C

ONCLUSION

An adaptive controller is proposed for the visual servo

system of a robot manipulator containing unknown camera parameters, kinematics and dynamics. The FAT is employed to cope with the time-varying uncertainties. The closed loop stability is proved by using the Lyapunov-like analysis. The realization of the proposed controller does not need to calculate the regressor matrix which is required in most adaptive designs for visual servo systems. Simulation results justify the performance of the proposed controller in fast tracking operations although most of the robot parameters are not available.

Fig.1 Visual servo system

600 650 700 750 800 850 900 950 1000

750 800 850 900 950 1000 1050 1100 1150 1200

X (pixels)

Y (pixels)

Fig.2 Tracking performance of end-point in the image space (—actual; --- desired).The real trajectory converges very quickly.

0 1 2 3 4 5 6 7 8 9 10

-10 -5 0 5

Time(sec)

X axis (pixels)

0 1 2 3 4 5 6 7 8 9 10

-60 -40 -20 0 20

Time(sec)

Y axis (pixels)

Fig.3 The tracking error. After some transient, the tracking error is very small, although we do not know precise kinematics and dynamics of the visual serving system.

0 1 2 3 4 5 6 7 8 9 10 -20

0 20 40 60 80

Time(sec)

torque 1 (N.M)

0 1 2 3 4 5 6 7 8 9 10

-10 -5 0 5

Time(sec)

torque 2 (N.M)

Fig.4 Control input torque.

R

EFERENCES

[1] S. Hutchinson, G.D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE Trans. Robot. Automa., vol.12, no.5, 1996, pp.651–670.

[2] P. Corke, and M. Good, “Dynamic effects in visual closed-loop systems,”IEEE Trans. Robot. Automa., vol.12,no.5, 1996, pp.671–683.

[3] B. Nelson and N. Papanikolopoulos, “Special issue of visual servoing,”

IEEE Robot. Automa. Mag., vol. 5, Dec. 1998.

[4] N. P. Papanikolopoulos and P. K. Khosla, “Adaptive robotic visual tracking: Theory and experiments,” IEEE Trans. Autom. Control, vol. 38, no. 3, 1993, pp. 429–445

[5] R. Kelly and A. Marquez, “Fixed-eye direct visual feedback control of planar robots,” J. Syst. Engineering, vol.4,no.5,1995, pp. 239–248.

[6] R. Kelly, “Robust asymptotically stable visual servoing of planar robots,”

IEEE Trans. Robot. Automa.,vol.12,no.5, 1996, pp. 759–766.

[7] G. D. Hager, W. C. Chang, and A. S. Morse, “Robot hand-eye coordination based on stereo vision,” IEEE Control Syst. Magazine, vol.

15, no. 2, 1995, pp. 30–39.

[8] J. Ma, J. Kosecka, and S. S. Sastry, “Vision-guided navigation for a nonholonomic mobile robot,” IEEE Trans. Robot. Automa., vol. 15, no. 3, 1999, pp. 521–536.

[9] R. Kelly, R. Carelli, O. Nasisi, B. Kuchen, and F. Reyes, “Stable visual servoing of camera-in-hand robotic systems,” IEEE/ASME Trans.

Mechatronics, vol. 5, no. 1, 2000, pp. 39–48.

[10] E. Malis and F. Chaumette, “Theoretical improvements in the stability analysis of a new class of model-free visual servoing methods,” IEEE Trans. Robot. Automa., vol. 18, no. 2, 2002, pp. 176–186.

[11] C. C. Cheah, M. Hirano, S. Kawamura, and S. Arimoto, “Approximate Jacobian control for robots with uncertain kinematics and dynamics,”

IEEE Trans. Robot. Automa., vol. 19, no. 4, 2003, pp. 692–702.

[12] F. Chaumette, “Image moments: A general and useful set of features for visual servoing,” IEEE Trans. Robot. Automa., vol. 20, no. 4, 2004, pp.

713–723.

[13] B. E. Bishop and M.W. Spong, “Adaptive calibration and control of 2D monocular visual servo systems,” Control Engineering Practice, 1999, pp.

423–430.

[14] E. Zergeroglu, D. M. Dawson, M. S. de Queiroz, and A. Behal, “Vision- based nonlinear tracking controllers with uncertain robot-camera parameters,” IEEE/ASME Trans. Mechatronics, vol. 6, no. 3, 2001, pp.

322–336.

[15] A. Astolfi, L. Hsu, M. S. Netto, and R. Ortega, “Two solutions to the adaptive visual servoing problem,” IEEE Trans. Robot. Automa., vol. 18, no. 3, 2002, pp. 387–392.

[16] M. R. Akella, “Vision-based adaptive tracking control of uncertain robot manipulators,” IEEE Trans. Robot. , vol. 21, no. 4, 2005, pp.748-753.

[17] Y. H. Liu, H. Wang, C. Wang, and K. Lam, “Uncalibrated visual servoing of robots using a depth-independent interaction matrix”, IEEE Tran.

Robot., Vol. 2, no. 4, 2006, pp. 804-817.

[18] H. Wang and Y. H. Liu, “Adaptive image-based trajectory tracking of robots,”in Proc. IEEE Int. Conf. Robot. Automa., 2005, pp.2564-2569.

[19] Y. H. Liu, H. Wang and D. Zhou, “Dynamic tracking of manipulators

using visual feedback from an nncalibrated fixed camera,” in Proc. IEEE Int. Conf. Robot. Automa., 2006, pp.4124-4129.

[20] H. Wang, Y. H. Liu and D. Zhou, “Dynamic visual tracking for manipulators using an uncalibrated fixed camera,” accepted by IEEE Trans. Robot., 2007.

[21] H. Wang and Y. H. Liu,” Uncalibrated dynamic visual tracking of manipulators,” in Proc. IEEE Int. Conf. Robot. Automa., 2007, pp.3198-3203.

[22] J. J. E. Slotine and W. Li, “On the adaptive control of robotic manipulators” Int. J. Robot. Res., vol.6, no.3, 1987, pp.49-59.

[23] C. C. Cheah, C. Liu, and J. J. E. Slotine, “Adaptive tracking control for robots with unknown kinematic and dynamic properties,” Int. J. Robot.

Res., vol. 25, no. 3, 2006, pp. 283-296.

[24] L. H. Hsu, R. R. Costa, and F. Lizarralde, “Lyapunov/passivity-bassed adaptive control of relative degree two MIMO systems with an application to visual servoing,” IEEE Trans. Autom. Control, vol.52, no.2, 2007, pp.364-371.

[25] A. C. Huang and Y. S. Kuo, ”Sliding control of nonlinear systems containing time-varying uncertainties with unknown bounds,” Int. J.

Control, vol.74, no. 3, 2001, pp.252-264.

[26] S. S. Ge, C. C. Hang, T. H. Lee and T. Zang, Stable Adaptive Neural Network Control, Boston: Kluwer Academic, 2001.

[27] J. T. Spooner, M. Maggiore, R. Ordonez and K. M. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems – Neural and Fuzzy Approximator Techniques, NY: John Wiley & Sons, 2002 [28] M. C. Chien and A. C. Huang, “Adaptive impedance control of robot

manipulators based on function approximation technique,” Robotica, vol.

22, no.4, 2004, pp. 395-403.

[29] A. C. Huang and Y. C. Chen, “Adaptive multiple-surface sliding control for non-autonomous systems with mismatched uncertainties,” Automatica, vol. 40, no.11, 2004, pp. 1939-1945.

[30] A. C. Huang and Y. C. Chen, “Adaptive sliding control for single-link flexible-joint robot with mismatched uncertainties,” IEEE Trans. Control Syst. Technol., vol. 12, no. 5, 2004, pp. 770-775.

[31] P. C. Chen and A. C. Huang,”Adaptive sliding control of non-autonomous active suspension systems with time-vary loadings,” J. Sound and Vibration, vol. 282, no. 3-5, 2005, pp. 1119-1135.

[32] P. C. Chen and A. C. Huang, “Adaptive Sliding Control of Active Suspension Systems with Uncertain Hydraulic Actuator Dynamics,”

Vehicle System Dynamics, vol. 44, no. 5, 2006, pp357-368.

[33] A. C. Huang, S. C. Wu, and W. F. Ting, “An FAT-based Adaptive Controller for Robot Manipulators without Regressor Matrix: Theory and Experiments,” Robotica, vol. 24, no.2, 2006, pp. 205-210.

[34] A. C. Huang and K. K. Liao, “FAT-based Adaptive Sliding Control for Flexible Arms, Theory and Experiments,” J. Sound and Vibration, vol.

298, no. 1-2, 2006, pp. 194-205.

[35] M. C. Chien and A. C. Huang, “Adaptive control of flexible-joint electrically-driven robot with time-varying uncertainties,” IEEE Trans.

Industrial Electronics, vol.54, no.2, 2007, pp.1032-1038.

[36] M. C. Chien and A. C. Huang, “Adaptive Control of Electrically-driven Robot without Computation of Regressor Matrix,” J. Chinese Institute of Engineers, vol.30, no.5, 2007, pp.855-862.

[37] L. E. Weiss, A. C. Sanderson, and C. P. Neuman, “Dynamic sensorbased control of robots with visual feedback,” IEEE J. Robot. Automa., vol. 3, no.

5, 1987, pp. 404–417.

[38] B. Espiau, F. Chaumette, and P. Rives, “A new approach to visual servoing in robotics,” IEEE Trans. Robot. Automa., vol. 8, no. 3, 1992, pp.

313–326.

[39] N. P. Papanikolopoulos, P. K. Khosla, and T. Kanade, “Visual tracking of a moving target by a camera mounted on a robot: A combination of vision and control,” IEEE Trans. Robot. Automa., vol.9, no.1, 1993,pp. 14–35.

[40] H. K. Khalil, Nonlinear Systems, Prentice-Hall, New Jersey, 2002.