Contents lists available atScienceDirect
Robotics and Autonomous Systems
journal homepage:www.elsevier.com/locate/robotLocation estimation for indoor autonomous vehicle navigation by
omni-directional vision using circular landmarks on ceilings
IChih-Jen Wu
a,1,
Wen-Hsiang Tsai
a,b,∗aDepartment of Computer Science, National Chiao Tung University, Hsinchu 30010, Taiwan
bDepartment of Computer Science and Information Engineering, Asia University, Taichung, 41354, Taiwan
a r t i c l e i n f o
Article history: Received 30 June 2006 Received in revised form 4 September 2008 Accepted 1 October 2008 Available online 1 November 2008 Keywords:
Location estimation Autonomous land vehicle Omni-directional vision Omni-directional camera Omni-directional image Circular-shaped landmark Indoor environment Ceiling a b s t r a c t
A novel approach to location estimation by omni-directional vision for autonomous vehicle navigation in indoor environments using circular landmark information is proposed. A circular-shaped landmark is attached on a ceiling and an omni-directional camera is equipped on a vehicle to take upward-looking omni-directional images of the landmark. This way of image taking reduces possible landmark shape occlusion and image noise creation, which come from the existence of nearby objects or humans surrounding the vehicle. It is shown that the perspective shape of the circular landmark in the omni-directional image may be approximated by an ellipse by analytic formulas with good shape-fitting effect and fast computation speed. The parameters of the ellipse are then used for estimating the location of the vehicle with good precision for navigation guidance. Both simulated and real images were tested and good experimental results confirm the feasibility of the proposed approach.
© 2008 Elsevier B.V. All rights reserved.
1. Introduction
Location estimation is essential for guidance of autonomous vehicles in many indoor navigation applications. A widely adopted approach is the vision-based technique. Most existing vision-based techniques deal only with frontal scenes acquired by traditional cameras and are easily interfered by unexpected objects around the vehicle. A feasible solution to this problem is to use an omni-directional camera [1–6] which looks upward at certain target shapes, called landmarks usually, attached on the ceiling [7]. This solution has the unique advantage of providing wide-angle views with fewer objects appearing in the field of view, thus reducing the guidance error coming from landmark occlusion, noise inference, etc. This is important for applications of intelligence robots such as cleaning robots, pet robots, tour guide robots, etc., which must work among humans or objects at close distances. On the other
I This work was supported financially by the Ministry of Economic Affairs under
Project No. MOEA 98-EC-17-A-02-S1-032 in Technology Development Program for Academia.
∗Corresponding author at: Department of Computer Science and Information
Engineering, Asia University, Taichung, 41354, Taiwan. Tel.: +886 4 23323456; fax: +886 4 23316699.
E-mail addresses:[email protected](C.-J. Wu),[email protected]
(W.-H. Tsai).
1 Tel.: +886 3 5131545; fax: +886 3 572 1490.
hand, even though obtaining the distance and orientation of the circular landmarks on the ceiling can be easily realized with a standard camera [8,9], a well-designed single omni-directional camera system may be used to replace several standard cameras so far as the image taking range is concerned.
In this study, the landmark is selected to be a circle. Analysis of circular shapes in omni-directional images is not well studied so far. It is found in this study that a circular shape, which becomes an irregular shape in an omni-directional image with no known shape descriptor, can be well approximated analytically by an elliptical shape. Consequently, it is appropriate to guide a vehicle equipped with an upward-looking omni-directional camera using a circular shape attached on a ceiling as a landmark, as is done in this study. Several merits can be identified in this approach, including: (1) the circular-shaped landmark attached on the ceiling is identically observable from every direction; (2) the circular shape, being elliptical when imaged, is easier to detect in low-resolution omni-directional images; (3) the elliptical shape provides more precise parameters for location estimation; (4) the elliptical shape does not get mixed up easily with other shapes found in the environment. Owing to these merits, stable and precise relative vehicle location estimation can be achieved for navigation. An illustration of the experimental navigation environment for this study, including a vehicle, a ceiling, and a landmark, is shown inFig. 1.
Most vehicle location estimation techniques using omni-directional images proposed up to now can be grouped into
0921-8890/$ – see front matter©2008 Elsevier B.V. All rights reserved.
Fig. 1. Relative positions of camera, ceiling, and circular landmark for providing
sufficient field of view and avoiding unexpected objects and humans appearing in acquired images.
three types: triangulation, full-scene matching, and mirror-lens
projection, in accordance with the way of image information. In
triangulation techniques [1,2], a standard location method is to identify surrounding landmarks in the environment and find their corresponding locations in the environment map built in advance. By the measured bearings of the landmarks [1], the location of an autonomous vehicle can be obtained. Yagi [2] correlated the angle of the vertical edge in an omni-directional image to the environment map to acquire the location of the vehicle. However, sometimes it is difficult to find natural landmarks which can be identified stably from omni-directional images.
In full-scene matching techniques [3,4], a vehicle locates itself by comparing images taken at its current location with reference images stored in its memory. Full-scene matching provides better feasibility by the use of omni-directional cameras. However, it takes much reference memory space to raise location precision. Gasper [3] showed that the position of the vehicle in the environment can be determined by comparing the vehicle’s current view with previously learned images, using a low-dimensional subspace of the input images obtained from a principal component analysis process. Menegatti [4] simplified the location problem by using the Fourier components of omni-directional images as the signatures of the acquired views.
The unique mirror-lens projection relation of the omni-directional camera also can be used to derive the location of an autonomous vehicle by 3-D computer vision techniques [5,6]. In Koyasu [5], the range information, which is obtained by an omni-directional stereo vision system composed of a pair of vertically-aligned omni-directional cameras, helps not only creating a 3-D map but also locating the vehicle itself. In [6], Cauchois compared synthetic landmark images with real landmark images taken from a calibrated omni-directional camera to accomplish absolute vehicle location works. However, these methods might be unstable or inapplicable for locating a vehicle in a space crowded with people or full of objects, such as in an exhibition room or in a library.
In the proposed approach, at first an upward-looking omni-directional camera on a vehicle is used to take an image of a circular-shaped landmark attached on the ceiling of an indoor space. An ellipse detection algorithm [10] is applied next to detect the projected shape of the landmark in the image. The irregular shape formed from the circular shape in the omni-directional image is approximated by an elliptical shape. The location of the landmark, including its distance and orientation, with respect to the camera on the vehicle are then derived analytically in terms of the major axis length and the center coordinates of the approximating ellipse. Finally, the move distance and the orientation change of the vehicle between two consecutive observations of the landmark are derived, which are useful for a number of autonomous vehicle applications.
The remainder of this paper is organized as follows. In Section2, we derive the analytic equations for approximating as an elliptical shape the irregular shape in an omni-directional image taken of a circular-shaped landmark. In Section 3, we describe how to estimate the vehicle location from the acquired image using the derived analytic equation, and show an application of the results to autonomous vehicle guidance. In Section4, some experimental results are described to show the precision and feasibility of the proposed method. Finally, some conclusions are given in Section5. 2. Approximation of irregular shape in omni-directional image taken of circular-shaped landmark by ellipse
The circular shape attached on the ceiling of the vehicle navigation environment for use as a landmark becomes irregular with no mathematical shape descriptor in an omni-directional image taken with a hyperboloidal omni-directional camera. We can approximate the irregular shape well by an ellipse, as mentioned previously, and this fact will be proved here. Specifically, an equation of the approximating elliptical shape in the image will be derived. The validity of this ellipse approximation will become clear in the derivation. The precision of the approximation will also be proved by some experimental results.
In Section 2.1, the projection transformation between the camera coordinate system and the image coordinate system will be described first. Then, the coordinate systems will be rotated horizontally to derive the equation of an ellipse in the image. In Section2.2a simulated shape of the circular landmark computed with the derived equation will be compared with the shape obtained by an imaging projection based on [5] to show the effectiveness of the proposed elliptical shape approximation.
2.1. Approximation of distorted circular shapes in omni-directional images by ellipses
The camera and image coordinate systems involved in this study using a hyperboloidal omni-directional camera are depicted inFig. 2, with their coordinates specified by (x
,
y,
z), and (u, v
), respectively. The hyperbolic shape of the omni-directional mirror in the camera coordinate system may be described as:R2 a2
−
z2 b2= −
1,
R=
p
x2+
y2.
(1)The focal point OMof the mirror is located at
(
0,
0, −
c)
and thecamera center OCat
(
0,
0, +
c)
, in the camera coordinate system,where c
=
√
a2
+
b2. The projection relationship between theimage coordinates (u
, v
) and the camera coordinates (x,
y,
z) canbe described as follows [5]: u
=
xf(
b 2−
c2)
(
b2+
c2)(
z−
c) −
2bcp
(
z−
c)
2+
x2+
y2,
v =
yf(
b2−
c2)
(
b2+
c2)(
z−
c) −
2bcp
(
z−
c)
2+
x2+
y2,
(2)where f is the focal length of the camera.
InFig. 2, let the circular-shaped landmark and its center and radius be denoted by W , Pw and Rw, respectively. And let the image of W in the hyperboloidal image be denoted by Q . Also, let (xw
,
yw,
zw) denote the camera coordinates of Pw. In this study, the normal vector of the landmark is assumed to be parallel with the optical axis of the camera. To simplify the derivation described later, we rotate, as shown in Fig. 3, horizontally the camera coordinate system and the image coordinate system through an angle ofθ
wdefined by:θ
w=
tan−1ywFig. 2. Coordinate systems involved in this study.
Fig. 3. Top view from the z direction showing the relationship between new and
original coordinate system with the new image coordinate system (u0
,v0
) obtained by rotating the u-axis through an angle ofθw=tan−1(y
w/xw)with respect to the center of the circular-shaped landmark W .
Then, the relation between the original camera coordinates (x
,
y,
z) and the resulting ones (x0,
y0,
z0) may be described by:x0
=
x cosθ
w+
y sinθ
w,
y0=
y cosθ
w−
x sinθ
w,
z0=
z (4) and the relation between the original image coordinates (u, v
) and the resulting ones (u0, v
0) may be described by:u0
=
u cosθ
w+
v
sinθ
w,
v
0=
u sinθ
w−
v
cosθ
w.
(5) Also, after this rotation, the circular shape of W in the new camera coordinate system may be expressed by:(
x0−
x0w)
2+
(
y0−
y0w)
2=
R2w,
z0=
zw0,
given that the center point Pwof W is located at (x0w
,
y0w,
z0w) withx0
w
=
xwcosθ
w+
ywsinθ
w,
y0w
=
xwsinθ
w−
ywcosθ
w,
z0 w=
zwaccording to(4). Notice that y0
w is now zero after the rotation according toFig. 3, so that the above equation becomes
(
x0−
x0w)
2+
y02=
R2w
,
z0=
zw0.
(6)Also, by the optical geometry of the camera described by(2), we have
v
0 u0=
y0 x0,
(7) or equivalently,v
0=
u 0 x0y 0.
(8) Equation(8)will be used in Section3.1.We are now ready to prove the previously-mentioned fact that the irregular shape of the circular landmark W appearing in the omni-directional image may be well approximated by an ellipse. After horizontally rotating the camera and the image coordinate systems for the angle of
θ
w=
tan−1(
yw
/
xw)
described by Eq.(3), (2)becomes u0=
x 0f(
b2−
c2)
(
b2+
c2)(
z0−
c) −
2bcp(
z0−
c)
2+
x02+
y02v
0=
y 0 f(
b2−
c2)
(
b2+
c2)(
z0−
c) −
2bcp(
z0−
c)
2+
x02+
y02.
(2A) In addition, the new y-coordinates y0wof the landmark circle center is 0. Then, by assuming that the horizontal distance from the origin of the camera coordinate system to the landmark is much larger than the radius of the landmark, we have y0
x0and the circle
(
x0−
x0w)
2+
(
y0−
y0w)
2=
R2w with y0w=
0 may be regarded relatively as a point which is its center located at (x0w
,
y0w) so that x02+
y02≈
x02 w+
y0 2 w=
x0 2w. Also, z0is a constant (denoted as hwnow) because the ceiling on which the landmark is attached is assume to be parallel with the camera coordinate system. As a consequence, the second equation in(2A)above for
v
0can be simplified tov
0=
y 0 f(
b2−
c2)
(
b2+
c2)(
z−
c) −
2bcq
(
z−
c)
2+
x02 w or equivalently, y0=
v
0M (9)where M is M
=
f(
b 2−
c2)
(
b2+
c2)(
z−
c) −
2bcq
(
z−
c)
2+
x02 w.
The other coordinate u0 of each shape pixel of the landmark
W may also be derived by approximation, but in a different way.
Under the same assumption mentioned above that the radius of the landmark is relatively very small with respect to the horizontal distance from the origin of the camera coordinate system to the landmark, the magnitude of x0of each shape pixel of the circular landmark W in the camera coordinate system is much larger than that of y0. Therefore, we may neglect the influence of the magnitude of y0in the computation of u0described by the first equation in(2A) so that
u0
=
x0
f
(
b2−
c2)
(
b2+
c2)(
z−
c) −
2bcp(
z−
c)
2+
x02and compute u0just in terms of x0. Regarding the above equation in the form u0
=
F(
x0)
, we may use the Taylor series to expand the function around x0wasu0
=
F(
x0) =
F(
x0w) + [(
x0−
x0w)/
1!]F
0(
x0w)
+ [
(
x0−
x0w)
2/
2!]F
00(
x0w) + · · · .
Ignoring the terms after the second, we have
u0
≈
F(
x0w) + [(
x0−
x0w)/
1!]F
0(
xw0) =
u0w+
(
x0−
x0w)
F0(
x0w).
(10) Equation(10)may be transformed easily intox0
≈
x0w+
(
u0−
u0w)/
F0(
x0w)
(11) with the first derivative F0calculated to be:F0
(
x0w) =
AE 1 BE−
CEx 0 w B2Ep
DE+
x0w!
,
(12) where AE=
f(
b2−
c2),
BE=
(
b2+
c2)(
z−
c) −
Cq
D+
x02 w,
CE=
2bc,
DE=
(
z−
c)
2,
z=
hw.
Now with x0 and y0 available, we come to the final stage of the derivation of the equation of the ellipse for approximating the distorted circular shape of the landmark in the omni-directional image. By substituting Eqs.(9)and(11)into Eq.(6)and rearranging the result, we can get
(
u0−
u0 w)
2 R2 wF0 x0w 2+
v
02M2 R2 w=
1 (13)which obviously specifies exactly an elliptical shape centered at (u0
w
,
0) with the lengths of the major and minor axes beingRwF0
(
x0w
)
and Rw/
M, respectively. This completes the proof. 2.2. Effectiveness of shape approximationTo check the effectiveness of the approximation of the circular shape of the landmark by the elliptical shape using Eq.(13), we show in Fig. 4 an example of the simulation results obtained in this study, in which both the original circular shape and the approximating elliptical one are drawn and superimposed on each
Fig. 4. Simulation of a series of circular shapes of the landmark at different places,
showing that the distorted landmark shape may be approximated well by ellipses. White pixels are simulated landmark shape points and black pixels are approximate ellipse points computed by Eq.(13). (a): illustration of the simulation results. (b): partially enlarged view of (a).
other for comparison: the former shape being drawn by Eq.(6)and then projected into the image plane by Eq.(2), and the latter being drawn directly by Eq.(13). The outer big circle in the figure marks the field of view of the camera. Inside the big circle, the original distorted circular shapes of W at different positions are drawn with white pixels, and the approximate elliptical shapes are computed using the coordinates of the white pixels and drawn by black pixels. From the figure, we can see that each black ellipse overlaps the corresponding white distorted circle quite well. This shows that the distorted circular shape of the landmark in the omni-directional image indeed may be approximated by the ellipse described by Eq.(13). This discovery offers great helps, as found in this study, in utilizing this kind of circular landmark to provide the location information for vehicle guidance, as described in the following section.
3. Vehicle location estimation
In this section we describe the proposed method for vehicle location estimation using the detected landmark as a known reference point. From each acquired image of the ceiling, we extract the circular landmark shape by image processing techniques, including thresholding, edge detection, and ellipse detection. For thresholding, since the artificially-made circular-shaped landmark contrasts well with the background in the image, a threshold value is selected to segment the distorted circular shape of the landmark in the image. And for edge detection in the image, the Sobel edge detector is applied. Since the landmark shape in the image can be regarded as an ellipse according to Section2, an ellipse detection algorithm [10] is employed to extract the landmark, yielding an approximating ellipse with its axis lengths computed.
3.1. Vehicle location estimation by axis lengths of an ellipse
We will derive here the location of the landmark, including its distance and orientation, with respect to the camera coordinate system on the vehicle for use in vehicle guidance. We derive first
Fig. 5. Top view from the z direction illustrating the relation between the axes
of the approximating ellipse and the horizontal distance of the circular-shaped landmark.
the horizontal distance of the landmark in terms of the lengths of the two axes of the approximating ellipse. As depicted inFig. 3, the x0axis is directed to the landmark center Pw0. For a better view of the involved situation,Fig. 3is redrawn asFig. 5in which d is the horizontal distance between the camera and the center of the landmark W to be derived; P0
αPβ0 is a diameter of W perpendicular to the x0axis; point Iα0 is the image of point Pα0; and pointIβ0 is the image of point Pβ0. It follows from Eq.(7)that
v
0 α u0 α=
y 0 α x0 α (14) where (u0α
, v
α0) are the coordinates of Iα0 in the (u0, v
0) coordinate system and (x0α
,
y0α) are the coordinates of Pα0 in the(
x0,
y0)
coordinate system. Similarly, we havev
β0 u0 β=
yβ0 xβ0 (15) where (u0β
, v
0β) are the coordinates of Iβ0 in the (u0, v
0) coordinate system and (x0β,
y0β) are the coordinates of Pβ0 in the (x0,
y0) coordinate system. Since P0αPβ0 is the diameter of W perpendicular to the x0axis, we have x0
α
=
x0βand so u0α=
u0β. Therefore, Eqs.(14) and(15)can be merged and rearranged to result inv
0 α−
v
β0 u0 α=
y 0 α−
y0β x0 α (16) where y0α−
y0βis the length of P0αPβ0 (denoted as
P 0 αPβ0
) which is
the known in advance;
v
α0−
v
0βis the length of I0αIβ0 (denoted as
I 0 αIβ0
) which is the major axis of the approximating ellipse in the
image plane; and u0
αis the u0coordinate of the center of the ellipse. The latter two parameters
I 0 αIβ0
and u 0
α can be computed right after the approximating ellipse is obtained. Also, as seen from the figure, x0
αis just the desired relative horizontal distance d between the camera and the center of the landmark W , or equivalently, the desired horizontal distance d of the landmark with respect to the vehicle. Therefore, we can derive from Eq.(16)the desired value of
d as d
=
P 0 αPβ0
I 0 αIβ0
×
u0α.
(17)Furthermore, using the ellipse makes it easier to solve the orientation
θ
w of the landmark with respect to the cameracoordinate system, or equivalently, with respective to the vehicle. Given that the center of the found ellipse in the image is located at (uw,
v
w), from Eq.(3)and the following equation derived from Eq.(7), yw xw=
v
w uw,
we can calculateθ
wbyθ
w=
tan−1 yw xw=
tan−1v
w uw.
(18)A merit of the above procedure is that there is no need of the vertical height value zw of the landmark with respect to the camera coordinate system in computing the distance d and the orientation
θ
w of the landmark. This provides an advantage of allowing dynamic changes of the camera height for the purpose of avoiding camera view occlusion by surrounding people or objects. Taking this advantage, we have included a shaft in the vehicle system for adjusting the height of the camera dynamically: if the landmark shape cannot be well extracted from the image taken at a certain camera height to yield an approximating ellipse, then the camera is lifted up automatically and gradually until a good result is obtained.Another merit of vehicle location estimation by Eqs.(17)and (18) is that the exact hyperbolic mirror shape of the omni-directional camera, as described in Eq.(1), need not be considered. This reduces the estimation error caused by using the possibly imprecise shape parameters of the mirror obtained by calibration, and simplifies the computation process involved in the estimation. Also, it provides a nature of generality of the proposed location estimation process using a given type of camera, so that it is unnecessary to re-design the location estimation process when a camera with a different hyperbolic mirror shape is used.
3.2. Estimation of vehicle move distances and orientation changes
In the previous section, we show how to estimate the distance and orientation of the landmark with respect to the vehicle. In this section we show how to find out, as an application of the previous results, the relative distance and orientation of the vehicle in a navigation cycle with respect to its location in the previous cycle, which we call the move distance and orientation change of the vehicle, respectively. Derivations of these parameters are useful for a number of autonomous vehicle applications, including recording of navigation paths, path planning for vehicle guidance, measurement of guidance errors, etc.
As shown inFig. 6(a) and (b), we denote the move distance of the vehicle between two consecutive observation times T1
and T2 as D. Using the approximating ellipse, we can, according
to Section3.1, calculate the horizontal distances d1, d2 and the
orientations
θ
1,θ
2of the landmark at T1and T2, respectively, withrespect to the vehicle. Based on the cosine theorem, we have
d21
+
D2−
2d1×
D×
cosθ
1=
d22,
which may be solved to get the desired move distance D as
D
=
d1cosθ
1−
q
d21cos2θ
1−
d21+
d 2 2 ifθ
2≤
90◦;
or (19) D=
d1cosθ
1+
q
d2 1cos2θ
1−
d21+
d22 ifθ
2>
90◦.
(20)Next, to find the desired orientation change of the vehicle, which we denote as
γ
, caused by a vehicle turning between the two consecutive observations, we acquire first, using Eq.(18), the orientations of the landmark with respect to the vehicle before and after the vehicle turning, denoted asθ
2andφ
, respectively, asdepicted inFig. 7(c) and (d). Then,
γ
can be computed easily byFig. 6. Illustration of the relative ALV location estimation. (a) and (b): the displacement D of the vehicle. (c) and (d): the orientation of the vehicle. 4. Experimental results
The effectiveness of the proposed location estimation method has been tested by some experiments conducted in this study, which include two parts: (1) using computer simulations to test if the circular shape of the landmark in the acquired images taken with omni-directional cameras with different shapes of hyperboloidal mirrors can be detected by the proposed ellipse approximation method; (2) using real images to determine the precision of the estimated vehicle locations relative to the landmark.
4.1. Simulations
In the first experiment of landmark shape approximation by ellipses, the first step was to create landmark shapes at different locations. A series of virtual circular-shaped landmarks with a radius of 20 cm were created and projected onto the image plane, in which the centers of these landmarks are located in a range of 300 cm in intervals of 50 cm with the orientation angles
θ
ranging from 0◦to 360◦in intervals of 5◦. Two kinds of common distortion, barrel and pincushion distortion as described in [11, 12], were added to make the simulation more realistic. In our simulations, the original image coordinates (u,v
) are inherently normal and distortion-free, as described by Eq.(2).Then, we add barrel and pincushion distortion to them by the following equations:
^
u
=
u+
u[k1(
u2+
v
2) +
k2(
u2+
v
2)
2];
^v = v + v[
k1(
u2+
v
2) +
k2(
u2+
v
2)
2]
where (u ,^
v
^) are the distorted image coordinates, and k1and k2aretwo parameters to control the shape of the geometric distortion. The coordinates (u ,^
v
^) then are taken as input to our method.Second, an ellipse detection algorithm [10] was used to detect these landmarks. Third, a rate of successful ellipse detections was computed. And at last, the three steps were repeated to compute the rates of successful ellipse detections for different kinds of omni-directional cameras which were created virtually only by changing the parameters of their hyperboloidal mirrors. More specifically, we changed only the mirror parameter b in (1) while maintaining the value of c invariant (
=
20 mm in our simulations) since the position and the focus of the cameras need not be changed. The smallest value of b was taken to be 146 mmTable 1
Rates of successful detections of landmark shapes taken by cameras with different shapes of hyperboloidal reflection mirror (c=20 mm).
b (unit mm) Detection rate (%)
146 100 147 100 148 100 149 100 150 100 151 100 152 100 153 100 154 100 155 100 156 100 157 100 158 95 159 <66
since otherwise the landmark image will exceed the range of the image plane. And the largest value of b was taken to be 158 mm because otherwise the rate of successful ellipse detection will drop dramatically owing to the shrinking of the landmark shape to a point in the image. The results are shown inTable 1. Examples of successful detections of ellipses in both original and distorted images are shown inFig. 7.
The simulation shows that the lens distortion has only a minor influence on the landmark detection. As indicated by Table 1, given shapes with proper sizes and reasonable image distortions such as the examples shown in Fig. 7, almost all landmarks can be successfully detected from the omni-directional images, even when the images are taken with cameras with different hyperboloidal mirrors.
4.2. Experimental results of real images for vehicle location estimation
The proposed method was also applied to two sets of real images. As shown in Fig. 8, both sets of images were taken with an upward-looking omni-directional camera mounted on an expandable vertical shaft on an autonomous vehicle. The relative positions of the camera, ceiling, and landmark are illustrated in Fig. 1. When performing location estimation, the shaft was lifted to raise the camera to the height of 1.4 m. Then, a landmark with a radius of 40 cm attached on the ceiling with a height of 2.9 m from the ground was imaged and its shape in the image detected. All
Fig. 7. Examples of successful detection of elliptical shapes. In (a)–(f), the elliptical shapes of simulated landmark images are marked by white pixels. Black pixels are
approximate ellipse points computed by the ellipse detection algorithm [10]. In (b)–(f), both horizontal and vertical grid lines are added in order to indicate the level of geometric distortion in each simulated image.
the images in the first set were taken from an identical orientation but with different distances from the camera to the landmark for the purpose of investigating the relation between the landmark distance and the estimation error. The results of this experiment are shown inTable 2. All the images in the second set such as those shown inFig. 9were taken from an identical orientation and at an identical location but with different camera heights for the purpose of investigating the relation between the depth variation with respect to the estimation. For measurement of the estimation precision, we define a distance error ratio and an orientation error as follows:
distance error ratio
=
(
real distance−
estimated distance)/(
real distance);
orientation error=
real orientation−
estimated orientation.
The distance error ratios and orientation errors computed from the two sets of images are shown inTables 2and3, respectively. The resolution of image is 640 by 480. The number of times of
experiments is five, that is, each distance or orientation value in TablesTables 2and3is the average of the data measured from the same orientation for five times. From the tables, we see that all the distance error ratios are smaller than 5% and that all the orientation errors are smaller than 2◦. Such estimation precisions are sufficient for the purpose of autonomous vehicle guidance and navigation purposes.
4.3. Identification and arrangement of the proposed landmarks in real applications
With a consideration of the integrity of a landmark-based navigation system, solving the identification and arrangement problems of the proposed landmarks is inevitable for our system. For example, how does the robot identify some landmark at which it is looking at a certain moment if it was taught the arrangement of landmarks in advance? In this section, we will discuss the identification and arrangement of the proposed landmarks in real applications.
Fig. 8. All the experimental images were taken by an autonomous land vehicle equipped with an upward-looking omni-directional camera. (a): the autonomous land
vehicle. (b): a close look of the camera on the vehicle.
Fig. 9. Two example images acquired by the camera equipped on the vehicle. (a) and (c): images acquired at different positions. (b) and (d): the enlarged images of the
Table 2
Error ratios in location estimations with landmarks located in the same directions but at different distances.
Estimated distance: d Real distance: d Estimated orientation Real orientation Distance error ratio (%) Orientation error (degree)
0.985 1.000 −80.8 −81.5 1.5 0.7 1.474 1.500 −81.4 −81.5 1.7 0.2 1.959 2.000 −83.0 −81.5 2.0 1.5 2.440 2.500 −83.0 −81.5 2.4 1.5 2.925 3.000 −82.8 −81.5 2.5 1.3 3.384 3.500 −81.7 −81.5 3.3 0.2 3.862 4.000 −80.9 −81.5 3.5 0.6 4.359 4.500 −81.6 −81.5 3.1 0.1 4.859 5.000 −81.1 −81.5 2.8 0.4 5.281 5.500 −81.6 −81.5 4.0 0.1 5.801 6.000 −81.8 −81.5 3.3 0.3 6.212 6.500 −81.1 −81.5 4.4 0.4 Table 3
Errors in location estimations with the landmarks located in the same directions and at the same distances but with different camera heights. Height of landmark from camera
(m) Estimated distance: d Real distance: d Estimated orientation Real orientation Distance error ratio (%)
Orientation error (degree)
2.5 3.076 3.000 −43.5 −42 2.5 −1.5 2.25 3.032 3.000 −44.2 −42 1.1 −2.2 2 3.074 3.000 −43.802 −42 2.5 −1.802 1.75 3.039 3.000 −41.845 −42 1.3 0.155 1.5 3.051 3.000 −40.569 −42 1.7 1.431 1.25 3.097 3.000 −42.466 −42 3.2 −0.466 1 2.942 3.000 −41.388 −42 −1.9 0.612
Fig. 10. Coloring scheme for identification of multiple circular shapes.
We solve the identification problem of individual landmarks by creating a color coding system. Each landmark is colored by two colors forming two concentric circles with each color being with one of nine pre-selected colors, as illustrated in Fig. 10. Furthermore, the nine colors are equally spaced in the hue attribute of the HSI color system, in order to make them more separable and reduce the influence of the lighting condition. Consequently, there are 81 different color combinations for use as the identification numbers to differentiate the landmarks in the map.
Moreover, since the location accuracy using our method becomes worse than 4% at distances of about 5 m, we suggest repeating the circular landmark on the ceiling at intervals of 10 m. Also, the repetitions should be done in two dimensions, forming a pattern of square grids with 10 m sides. The diameter of the circular landmark is 80 cm for a ceiling at a height of 2.9 m. If the ceiling height is changed, then the size of the circle may be changed proportionally.
5. Conclusion
In this study, a new approach to location estimation of an autonomous vehicle for navigation guidance in an indoor environment using a circular-shaped landmark on the ceiling by omni-directional vision techniques has been proposed. It is proved both by theoretical derivations and experimental results that the distorted landmark shape appearing in the image may
be well approximated, based on the application of Taylor series expansion, by an ellipse. After the elliptical shape is extracted by an ellipse extraction algorithm, a location estimation approach is proposed by using the ellipse parameters for autonomous vehicle applications.
The proposed location estimation approach with a unique ability of allowing dynamic camera height changes increases the flexibility of creating an occlusion-free camera view in unknown complex environments with obstacles. Additionally, the exact hyperbolic mirror shape of the omni-directional camera need not be considered which reduces the estimation error caused by the use of the imprecise shape parameters of the mirror obtained from calibration. The computation of the proposed approach is analytic, thus speeding up the estimation process and so reducing the navigation cycle time. Both simulated and real images were tested and good experimental results prove the effectiveness of the proposed approach.
Acknowledgment
This work was partially supported by Technology Development Program of Academia, Ministry of Economic Affairs, Taiwan, R. O. C., under Grant No. 98-EC-17-A-02-S1-032.
References
[1] M. Betke, L. Gurvits, Mobile robot localization using landmarks, IEEE Transactions on Robotics and Automation 13 (2) (1997) 251–263.
[2] Y. Yagi, Y. Nishizawa, M. Yachida, Map-based navigation for a mobile robot with omni-directional image sensor copis, IEEE Transactions on Robotics and Automation 11 (5) (1995) 634–648.
[3] J. Gaspar, N. Winters, J. Santos-Victor, Vision-based navigation and environ-mental representations with an omni-directional camera, IEEE Transactions on Robotics and Automation 16 (6) (2000).
[4] E. Menegatti, T. Maeda, H. Ishiguro, Image-based memory for robot navigation using properties of the omni-directional images, Robotics and Autonomous Systems 47 (4) (2004) 251–267.
[5] H. Koyasu, J. Miura, Y. Shirai, Recognizing moving obstacles for robot navigation using real-time omni-directional stereo vision, Journal of Robotics and Mechatronnics 14 (2) (2002) 147–156.
[6] C. Cauchois, E. Brassart, B. Marhic, C. Drocourt, An absolute localization method using a synthetic panoramic image base, in: Proceedings of IEEE Workshop on Omnidirectional Vision, Copenhagen, Denmark, June 2002, pp. 128–135. [7] C. Becker, J. Salas, K. Tokusei, J.-C. Latombe, Reliable navigation using
[8] Y. Ogawa, J.H. Lee, S. Mori, A. Takagi, C. Kasuga, H. Hashimoto, The positioning system using the digital mark pattern-the method of measurement of a horizontal distance, Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (1999) 731–741.
[9] S.J. Ahn, W. Rauh, M. Recknagel, Circular coded landmark for optical 3D-measurement and robot vision, Proceedings of International Conference on Intelligent Robots and Systems (1999) 1128–1133.
[10] C.T. Ho, L.H. Chen, A high-speed algorithm for elliptical object detection, IEEE Transactions on Image Processing 5 (3) (1996) 547–550.
[11] L. Ma, Y. Chen, K.L. Moore, Flexible camera calibration using a new analytical radial undistortion formula with application to mobile robot localization, Proceedings of the IEEE International Symposium on Intelligent Control (2004) 799–804.
[12] Z. Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (11) (2000) 1330–1334.
Chih-Jen Wu received the B.S. degree and the M. S.
degree, both in engineering science from National Cheng Kung University, Tainan, Taiwan, in 2000 and 2002, respectively.
Mr. Wu worked as a research assistant in the Laboratory of System Integration in National Cheng Kung University from August 2000 to July 2002, and as a research engineer in the Computer Vision Laboratory in National Chiao Tung University since August 2002 till now. He is also a Ph. D. student in the Department of Computer Science at National Chiao Tung University since 2002. His current research interests include computer vision, robotics, pattern recognition, and their applications.
WHsiang Tsai received the B. S. degree in electrical
en-gineering from National Taiwan University in 1973, the M. S. degree in electrical engineering from Brown University in 1977, and the Ph. D. degree in electrical engineering from Purdue University in 1979. Dr. Tsai joined the fac-ulty of National Chiao Tung University (NCTU) in Taiwan in November, 1979, and was an NCTU Chair Professor in the Department of Computer and Information Science. Since August, 2004, he has been the President of Asia University in Taiwan.
At NCTU, Professor Tsai was the Head of the Depart-ment of Computer and Information Science from 1984 to 1988, the Dean of General Affairs from 1995 to 1996, the Dean of Academic Affairs from 1999 to 2001, and Vice President from 2001 to 2004. He served as the Chairman of the Chinese Image Pro-cessing and Pattern Recognition Society of Taiwan from 1999 to 2000. He has been the Editor of several academic journals, including Journal of the Chinese Engineers, International Journal of Pattern Recognition and Artificial Intelligence, Journal of Infor-mation Science and Engineering, and Pattern Recognition. He was the Editor-in-Chief of Journal of Information Science and Engineering from 1998 to 2000.
Professor Tsai has received many awards, including one Distinguished Research Award, four Outstanding Research Awards, two Special Researcher Awards, and one Outstanding Researcher Award, all from the National Science Council, between 1987 and 2001. He was the recipient of the 13th Annual Best Paper Award from the Pattern Recognition Society of the U. S. A. He also received the Academic Award of the Ministry of Education in 2002. Finally, he was the recipient of the ISI Citation Classic Award in 2001.
Professor Tsai’s major research interests include image processing, pattern recognition, computer vision, virtual reality, and information copyright and security protection. So far he has published 311 academic papers, including 121 journal pa-pers. Dr. Tsai is a senior member of IEEE and is currently the Chair of the Computer Society of IEEE Taipei Section in Taiwan.
