Image-based fuzzy control system
G.-M. Chen, P.-Z. Lin, W.-Y. Wang, T.-T. Lee
and C.-H. Wang
A novel image-based fuzzy control (IBFC) scheme is developed to imitate the way humans use visual information to control objects. A CCD camera gathers images of the controlled plant, and a simple algor-ithm analyses the images. The proposed image analysis algoralgor-ithm utilises image information more intuitively than visual servo control systems. The difference between a reference image and the current image is numerically expressed and directly used by a fuzzy control system using a human-like control law. To investigate the effectiveness of the proposed IBFC scheme, it is applied to control an inverted pen-dulum system. Simulation results show that the IBFC system can achieve favourable tracking performance without prior knowledge of the controlled plant.
Introduction: For the past decade, machine vision has attracted a lot of research, and is widely used in many industries, especially the robot and vehicle industries[1, 2]. It mimics the human sense of vision and allows for non-contact measurement of the environment. A control scheme which aims at using the information provided by machine vision to control the motion of robot or vehicle systems is called visual servo control, and is usually divided into two classes, position-based visual servo (PBVS) and image-based visual servo (IBVS)[3, 5]. The perform-ance of PBVS control highly depends on the accuracy of image feature extraction and camera calibration. On the other hand, co-ordinate trans-formation in the IBVS, involving calculation of the image Jacobian, adds significant complexity to the system design as well as inducing a heavy computational burden.
Considering the drawbacks resulting from the ways that the image-type information is used in PBVS and IBVS, we decided to use the image-type information in a more intuitive way. By using the proposed algorithm, we can ‘read’ the images and obtain knowledge of the differ-ence between referdiffer-ence and current images. Then, the image differdiffer-ence is directly imported into a fuzzy logic control system to derive a suitable control law. Hence, we do not need to reconstruct the 3D environment to extract information such as position and velocity, and complex co-ordination transformation is not necessary.
Image analysis algorithm: Using a CCD camera, we take an image of the controlled object in a static reference position (called the reference image), and images of the controlled object (called the current images) during the control process. We use a simple algorithm to numerically express the differences between the reference image and the current image. The fuzzy control rules are determined by an expert. A simple example of an inverted pendulum is used to illustrate the proposed algorithm.Figs. 1aandbshow the reference and current images of the rod of the inverted pendulum, respectively. For simplicity, the image resolution is chosen as 10V 10H; however, the developed algorithm can be applied to analyse images with arbitrary resolution. The image to be analysed is first translated into a transformation matrix, M [ Rnmas follows: M ¼ m11 m12 m1 m m21 m22 m2 m .. . .. . . . . .. . mn1 mn2 mnm 2 6 6 6 4 3 7 7 7 5i ¼ 1; 2; . . . ; n; j ¼ 1; 2; . . . ; m where nV mH is the image resolution, mij¼ 1 if the corresponding
pixels are partially or fully covered by the object in the image, and mij¼ 0 if the corresponding pixels are not covered. The transformation
matrices ofFigs. 1aandbare defined as Mrand Mc, and are shown in
the right sides ofFigs. 1aandb.
Define Me¼ Mr2 Mc. To confirm the difference between the
refer-ence and current images in the vertical direction, we first determine a vertical weighting vector v ¼ [v1v2. . .vn] [ Rn, where 0 , v1,. . .
, vn(v ¼ [1 2. . .10] for this example). Then, we define Me,v¼ vMe.
The Frobenius norm of Me,vis calculated as kMe,vkF¼
p (Pi¼110 P j¼1 10 jmijj2) ¼ p
(88). It is clear that a large kMe,vkFimplies a big difference
between the reference and the current images in the vertical direction, and a small kMe,vkFimplies a small difference.
We are also interested in the image information in the horizontal direction. That is, if the controlled object in the static position is
viewed as a reference point, we would like to confirm if the object during the control process is located to the left or right. Among the elements which equal 1 in the transformation matrix M, we denote the leftmost and rightmost ones as mland mr, respectively. Suppose that
ml is in jlth column and mr is in the jrth column, then, a ‘middle
index’ is defined as I ¼ ( jlþjr)/2. Thus, the middle indices of Mr
and Mc, denoted as Irand Ic, respectively, are Ir¼ (5 þ 6)/2 and Ic¼
(5 þ 8)/2. It is easy to see that Ir. Ic implies that the controlled
object is located on the left side of the object at the equilibrium, and vice versa. Combining the image information in both the vertical and horizontal direction, we can numerically express the difference between the reference and current images as
z ¼ skMe;vkF; s ¼
1; if Ir, Ic
1; if Ir. Ic
Thus, we can easily express the difference betweenFigs. 1aandbas z ¼p(88). 0º 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 a b 30º 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Mc = Mr =
Fig. 1 Examples of images of rod of inverted pendulum
Remark 1: That the camera field of view should completely cover the motion of the controlled object.
Remark 2: High resolution is indispensable for precise image analysis but is computationally expensive. However, the image analysis per-formed with the proposed algorithm can achieve the necessary precision with a quite low resolution, e.g. 10V 10H inFigs. 1aandb. This will be further examined in the Section ‘Results’.
Remark 3: The choice of the vertical weighting vector v could influence the precision of the image analysis in the vertical direction, especially for low-resolution images.
Before constructing the proposed image-based fuzzy control (IBFC) scheme, we first formulate the control problem. Consider a general nth-order nonlinear system of the form x(n)¼ f (x, u), where x ¼ [x x. . . . x(n21)
]T is the state vector of the system (which can be left unknown in our system), f (x,u) is the unknown nonlinear system dynamics and u is the control input of the system. Since the system dynamics f (x,u) cannot be exactly obtained, the considered system poses an interesting and challenging problem for control systems. The model-free feature of fuzzy control provides a feasible solution to this problem. Fig. 2 shows the block diagram of the proposed IBFC scheme.
FC system F dt d u × Me,v z s controlled plant matrix transformation CCD camera q − + Mr Mc Mr Mc reference image current image
Fig. 2 Block diagram of IFBC scheme
Results: A simulation of an inverted pendulum is provided as an example to verify the effectiveness of the proposed IBFC scheme. The control target is to balance the rod in the upright position. To obtain a high-contrast image, we place the inverted pendulum against a white background.
The dynamics of the inverted pendulum are defined in[6]. The input of the fuzzy system is defined as z ¼ [z z.], where z.is the time derivative of z. The simulation begins with the rod tilted at a 358 angle, and a time derivative of zero. The reference trajectory is that to keep the rod in the upright position. For this simulation, rather than generating the images on the fly, we created predefined images based on the rod angle (u). We variedufrom 2908 to 908, with a quantisation of 58 for a total of 37 images. The higher the resolution, the smaller the quantisation level should be. It should be emphasised that the quantisation and off-line matrix transformation are only necessary for the computer simu-lation and can be skipped in a practical implementation.
The simulation results are shown inFig. 3, where Figs. a and b show the tracking response ofuandu.. InFig. 3a, we see thatudoes not con-verge to zero but oscillates aroundu¼ 2.58 (half of the quantisation level), and inFig. 3bthat the oscillation ofudirectly causes the oscil-lation ofu. around zero. This phenomenon clearly shows the influence of quantisation. If the resolution of the analysed image is higher, a smaller quantisation can be chosen and thus both u and u. will be closer to zero. ro d an g le q , deg ti m e de ri va ti ve of r o d a n g le q , de g /s 2 time, s 40 35 30 25 20 15 10 5 0 –5 –10 –15 –20 –25 300 200 100 0 –100 –200 –300 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 time, s a b 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 .
Fig. 3 Simulation results
Conclusion and future work: We propose a novel image-based fuzzy control scheme (IBFC). A CCD camera is substituted for human eyes, and the images captured by the CCD camera are analysed by the proposed simple algorithm. The algorithm provides an intuitive and
reasonable way to deal with the image information. Moreover, complex co-ordination transformations can be avoided. The simulation results demonstrate that even under the limit of low image resolution, it can achieve favourable tracking performance on an inverted pendulum. Having made some progress in this study, we still have some difficult problems to be solved, e.g. the lack of stability analysis may harm the reliability of the system, and the proposed image analysis algorithm is not general enough. In future, we will concentrate our work on exploring stability analysis and removing the restrictions of the proposed algorithm, making it more reliable and applicable for practical systems.
#The Institution of Engineering and Technology 2008
10 January 2008
Electronics Letters online no: 20080057 doi: 10.1049/el:20080057
G.-M. Chen, P.-Z. Lin and C.-H. Wang (Department of Electrical and Control Engineering, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu 300, Taiwan)
T.-T. Lee (Department of Electrical Engineering, National Taipei University of Technology, Taipei, Taiwan)
W.-Y. Wang (Department of Applied Electronics Technology, National Taiwan Normal University, Taipei, Taiwan)
E-mail: [email protected] References
1 Trivedi, M.M., Gandhi, T., and McCall, J.: ‘Looking-in and looking-out of a vehicle: computer-vision-based enhanced vehicle safety’, IEEE Trans. Transport. Syst., 2007, 8, (1), pp. 108 – 120
2 Rousseau, P., Desrochers, A., and Krouglicof, N.: ‘Machine vision system for the automatic identification of robot kinematic parameters’, IEEE Trans. Robot. Autom., 2001, 17, (6), pp. 972 – 978
3 Hutchinson, S., Hager, G.D., and Corke, P.I.: ‘A tutorial on visual servo control’, IEEE Trans. Robot. Autom., 1996, 12, (5), pp. 651 – 670 4 Mezouar, Y., and Chaumette, F.: ‘Path planning for robust image-based
control’, IEEE Trans. Robot. Autom., 2002, 18, (4), pp. 534 – 549 5 Cho, J.S., Kim, H.W., and Kweon, I.S.: ‘Image based visual servoing
using position and angle of image features’, Electron. Lett., 2001, 37, (13), pp. 862 – 864
6 Wang, W.Y., Cheng, C.Y., and Leu, Y.G.: ‘An online GA-based output-feedback direct adaptive fuzzy-neural controller for uncertain nonlinear systems’, IEEE Trans. Syst., Man Cybern., B, 2004, 34, (1), pp. 334 – 345
