A Study on Automatic Indoor Navigation Techniques for Vision-Based Mini-Vehicles with Off-Line Environment Learning
Capability
Chung-Chi Lai (
) and Wen-Hsiang Tsai (
) Department of Computer & Information Science
National Chiao Tung University, Hsinchu, Taiwan 300, R. O. C.
Tel: 886-3-5712121 Ext. 56650, e-mail: gis90576@cis.nctu.edu.tw Abstract
An effective vision-based approach to autonomous vehicle navigation in indoor corridors using off-line learning and 2D image analysis techniques is proposed. A computer- controlled mini-vehicle is constructed as a test bed for this study. To achieve flexible navigation in different environments, an effective method for navigation path learning using effective path features and adaptive off-line learning processes is proposed first. And to navigate in various corridor paths, three vehicle guidance strategies for four different kinds of corridor sections in buildings are proposed next. The proposed techniques have been tested on the mini-vehicle and successful learning and navigation sessions show the feasibility of them.
1. Introduction
For flexibility and stability, many applications of vehicle navigation in indoor environments require learning of the navigation path before navigation sessions can be started. In Martinez and Vitria [1], a user has to drive manually a robot around the environment to conduct learning and record the navigation path at proper locations. After learning, the user has to select some spots along the navigation path and label them with numbers. Then, the navigation system can receive the user’s command and navigate to every labeled position. In Davison and Murray [2], a vehicle used in their
experiment was trained to recognize certain landmarks. Many landmarks were placed manually in the navigation environment to provide navigation directions. The vehicle can detect the landmarks and follow them to reach the goal. Furthermore, in Hayet, Lerasle and Devy [3], a vehicle was designed to identify obvious square blocks as landmarks in the learning stage, and record the size and color of these landmarks as well as the positions where the vehicle sees these landmarks. The vehicle can navigate forward by detecting landmarks and checking them on size and color one by one. Several other methods for learning navigation paths can be found in [4-6].
On the other hand, many navigation methods were developed in recent years. In Desouza and Kak [7], a survey of mobile robot navigation techniques based on computer vision is given. The survey includes two subjects about navigations in structured and unstructured environments. In structured indoor environments, navigations according to map-based approaches or by tracking landmarks are discussed. In unstructured indoor environments, navigations using optical flows and by recognition of specific objects in the environment are discussed. In Kyoung, Lee and Kweon [8], a robot was designed to detect the right and the left baselines and generate a risk area between the baselines.
The robot can navigate in the risk area and avoid
obstacles. In [9-11], the vehicle keeps navigating on the central path of a route. Techniques of detecting baselines in corridor sections and computing the center points of corridors were proposed in these studies.
In this study, we try to design a back-and-forth navigation technique in a certain corridor environment using a mini-vehicle.
Besides, a learning method that processes the learned data without stopping or intervening by users is proposed. The proposed approach for navigation with a mini-vehicle in this study consists roughly of three stages. The first stage is navigation path learning. For this purpose, the mini-vehicle is controlled to record features along a pre-selected navigation path by a wireless-controlled notebook PC. The features are classified into two types, including the action type and the feature type, according to their characteristics. These features are recorded as a data list for the later process.
The second stage is learned data processing. For the purpose of developing a learning system, named as off-line learning system, a method for processing learned data is proposed. With this method, a user may control the mini-vehicle along a path once at his/her will and the recorded data can be transformed into a meaningful data list for use in the navigation stage. Besides, the user does not have to know any professional technique except some control rules for learning.
The third stage is back-and-forth navigation. Three strategies are proposed to navigate in four different types of sections in corridor environments. The first strategy, named as line following, is used for the mini-vehicle to navigate in straight sections. The second, named
as curve following, is used for turning left at corners with a parabolic model. The third one is called dead reckoning, which is utilized for navigating in environments without features, especially for crossing corridors.
2. System Configuration
A commercial motor-driven mini-vehicle, as shown in Fig. 1, is adopted as a test-bed of the proposed approach. This mini-vehicle is modified in this study by adding a web camera, some control circuits, a microcomputer, a pair of 7.2-volt batteries, and a notebook PC. The mini-vehicle has two driving wheels at the two sides and two casters in the front and the rear of the vehicle. The driving wheels are controlled by two stepping motors, respectively. The vehicle body is 22 cm in length and the width between the driving wheels is 18.6 cm. Above the driving wheels is a circuit board with an 8051 microcomputer, an RS232 port and some circuits connected to the motors. There is an aluminum box above the circuit board in which we place a notebook. The web camera is fixed on the front part of the aluminum box. The height from the ground to the top of the aluminum box is 21 cm.
The batteries are placed at a position between the front caster and the aluminum box.
Fig. 1 The vehicle used in this study.
3. Off-line Learning Strategies for Indoor Navigation
The purpose of learning in this study is to
gather appropriate features of the environment for smooth and stable guidance, and to allow the vehicle the mobility to navigate in the environment at will. The proposed learning procedure consists of two major algorithms, called manual off-line learning algorithm and automatic off-line learned data processing algorithm. The goal is to let a user control the mini-vehicle along a desired path just once without stopping at any location for manual keeping of environment parameters.
3.1 Manual Off-line Learning Algorithm In this algorithm, some schemes are proposed to accomplish the manual learning by a user. The proposed control module consists of two parts: the use of function keys and the wireless control. Some function keys are defined for users to control the mini-vehicle by pressing these keys in learning. These function keys are defined as follows.
Table 1. The table of function keys.
Function keys Actions up arrow Move advance down arrow Move backward
left arrow Turn leftward right arrow Turn Rightward
q Stop
space Stop learning By the constraint of the size of the mini-vehicle, a remote control method is proposed to drive the mini-vehicle. Because the notebook PC on the mini-vehicle (called the on-vehicle PC) is equipped with a wireless LAN card, a user can communicate with the on-vehicle PC by other computers with network capability.
In this study, we control the mini-vehicle with another notebook PC (called the off-vehicle PC) with a wireless LAN card. The communication between the on-vehicle PC and the off-vehicle PC is accomplished through the use of a program
named “WinVNC.” By this communication channel, a user can press a key on the off-vehicle PC in his hand just like pushing down the corresponding key on the on-vehicle PC.
With the characteristics of being obvious, easy to extract, and stable, path features are the most useful information in the process of learning a navigation path. In the proposed method, features employed for use in the learning stage include commands of actions and turn angles, the distance between two turning points, and the safe distance to the corridor wall.
A. Learning of Actions and Turn Angles of the Vehicle
The mini-vehicle is controlled by a user in the learning stage in an unceasing way using continuous commands until the user decides to stop learning. Therefore, the command list is useful information about the navigation path.
User’s commands, including actions and turn angles, are recorded sequentially in accordance with the corresponding function keys pressed by the user.
According to their types, the actions of the mini-vehicle are recorded in terms of several symbols. A list of the symbols for recording actions and turn angles are shown in Table 2.
Table 2. The table of recording actions and turn angles.
Actions and turn angles Record symbols
Move advance a
Move backward b
Turn leftward l
Turn Rightward r
Stop s
Degrees of turn angles Degrees/2.5 B. Learning of Distance Between Two Turning
Points
When the mini-vehicle runs in a straight section, the program in the 8051 single-chip
microprocessor will record the step numbers of the stepping motors and report this information to the on-vehicle PC. This is accomplished by turning on the odometer program at one turning point and turning it off at another so that the number n of steps between two turning points can be computed. The value of n will be regarded as the length of this straight section because the corresponding real-world distance d can be calculated with n according to the following function:
d = (n / 200) × 2 × 4.3 ×. (4.2)
The number 4.3 means the radius of the wheels. The value n will be recorded to represent the length of a straight section following a recorded character “a” which means moving straightly in a certain section.
C. Learning of Safe Distance Between Navigation Path and Baseline
Because the mini-vehicle is constrained to navigate along the right baseline, it is very important to know the distance between the mini-vehicle and the right baseline. In the learning stage, a user has to control the mini-vehicle near the right baseline and a safe distance is calculated. But it is not easy to keep the safe distance constant during the learning process due to the manual adjustments through the navigation path. Therefore, the average of the safe distance of the straight section is calculated.
The average value will be saved below the character “a” for use in the navigation stage.
3.2 Automatic Off-line Learned Data Processing Algorithm
In this section, an automatic off-line learned data processing algorithm is proposed. In this algorithm, a learned data processing scheme is designed to decide the navigation strategy of
every corridor section for transforming the learned data into a list of meaningful information automatically. And then the mini-vehicle can navigate by the list without users’ operations.
In order to achieve the goal of off-line learning, the learning program must have the capabilities to identify from the learned data the various types of corridor sections. Therefore, four methods for identifying learned data types are proposed to achieve the goal of off-line learning.
Finally, an integrated method is proposed to analyze the learned path and decide the navigation strategy of every corridor section.
A. Identification of Corners for Turning by Curve Following
In this case, some rules are defined to find the learned data that represent corners in the environment. In the learning stage, to navigate around the corner a user has to control the mini-vehicle to turn an anglea left, go straight for a certain distance, and turn again another angleb left. The learned path is illustrated in Fig. 2, where P1 and P2 are the two turning points and d is the length of the straight section.
A threshold value ta is defined to check the learned data. If the learned data conform with the rule:
|a +b900| ta,
these learned data represent a corner because if the mini-vehicle goes around a corner, there will be two connective left turns and the sum of their degrees is almost 90 degrees. The entire set of learned data will be scanned to find subsets of data that match this rule, and then convert them into the corner type. In the example in Fig. 2, at first, the relative coordinates of P2 with respect to P1 in the real-world coordinates are calculated in terms of the distance d. The coordinates of P2 are:
P2(x2, y2) = (d sina, d cosa).
Then the learned data is converted into a character “c” and the coordinates of P2.
Fig. 2 An example of learned data of the corner.
B. Identification of Straight Sections for Dead Reckoning
In this section, some rules will be defined to find the straight sections where the mini-vehicle has to navigate with the dead reckoning method in the navigation stage. An example is the straight section at a corner which is perpendicular to the corridor and has no right-side baseline or any other obvious feature to utilize for navigation. For this purpose, two threshold values tb and tc are defined to check the learned data. More specifically, if the learned data conform with the following rules:
|a900| tb,
|b900| tb,
|da + db + d – (corridor width)| tc, these learned data will be converted into the character “d” and recorded, and the mini-vehicle will navigate in this section with dead reckoning in the navigation stage.
C. Identification of Straight Sections for a Hybrid Strategy
In this section, we describe the proposed rules for identifying in the learned data a straight section in which the mini-vehicle will navigate by line following and dead reckoning. As illustrated in Fig 3, the mini-vehicle navigates in a straight section with line following until the right baseline disappears, and then goes forward for a certain
distance with dead reckoning. A reason for using this hybrid method for vehicle guidance is that the mini-vehicle has to turn right at a corner, and it is impossible to reach the turning point by using the line following method only. That is, the mini-vehicle has to navigate with dead reckoning when all stable features disappear after line following.
For this purpose, the learned data of the section where the mini-vehicle has to run using the hybrid method in the navigation stage should be found out and marked with a special character
“h”. In the learned data, right-turning actions are picked out to be a turning action which should be guided by the hybrid method. We then mark the straight section, which is in front of the right-turning action, to navigate with the line following method and the dead reckoning method in the navigation stage.
Fig. 3 The concept of turning with a hybrid method.
D. Identification of Straight Sections for Line Following
Because the learned data are scanned sequentially and compared with the rules for the above three types of sections in turn, the remaining data of straight sections are just those with the stable features of right baselines. The mini-vehicle can navigate in these sections by line following.
E. Process of Off-line Learning
By integrating these identifying methods, a procedure of learned data processing is proposed to decide the navigation strategies for the learned
corridor sections. The algorithm of learned data processing is described as follows.
Algorithm 1. Deciding navigation strategies by learned data processing.
Step 1. Scan the learned data and check the turning angles of neighboring turning points.
Step 2. Find corner sections where the mini-vehicle should navigate with curve following when the sum of two neighboring angles is approximately equal to 90 degrees.
Step 3. Find straight sections where the mini-vehicle should navigate with dead reckoning when the two neighboring angles are both approximately equal to 90 degrees.
Step 4. Find straight sections where the mini-vehicle should navigate with line following and dead reckoning when there is a right-turning action at the end of each of these straight sections.
Step 5. Repeat Steps 2~4 until the entire learned data are scanned and processed. The remaining learned data are decided to be the straight sections to navigate with the line following method.
After the learned data are processed, the original learned data will become a new data list which contains the navigation information. An example of navigation path with marked navigation strategies is shown in Fig. 4.
4. Back-and-Forth Vehicle Navigation in Indoor Corridor Environment
In this charter, the proposed vehicle navigation method is described. It is desired to guide the mini-vehicle to navigate in the corridor environment where the mini-vehicle has learned
before and return the starting point for closed-loop navigation. For these purposes, three strategies for navigation in straight corridor sections and corners are proposed in this study.
They are line following, curve following, and dead reckoning. According to these strategies, the mini-vehicle can navigate in four kinds of corridor situations in the corridor environment based on the learned data. The processed learned data will be scanned one by one and the mini-vehicle will implement these actions sequentially in the navigation stage.
Fig. 4 An example of navigation strategies in every learned section.
4.1 Navigation Strategy for Straight Corridor Sections
When the mini-vehicle navigates in a straight section, the right baseline is the most important feature for navigation because it is very stable and represents the moving direction clearly.
Therefore, a line following strategy is used for the mini-vehicle to navigate in the straight section according to the right baseline. The strategy of line following is selected if the character “a” is scanned from the learned data in the navigation stage. The safe distance is the key feature to perform line following. The concept of implementing the line following strategy is illustrated in Fig. 5.
4.2 Navigation Strategy for Turning Left at Corners
If the mini-vehicle arrives at a turning point in the navigation stage and receives the learned
data “c” at the same time, a curve following method is used for the mini-vehicle to turn left along a parabolic curve path. It is important to know the equation of the parabolic curve E because it is the navigation path for the vehicle to turn around a corner.
In order to calculate the equation E, the points of the parabolic curve have to be found. As mentioned before, the coordinates of the turning point are defined to be P1(x1, y1), where x1 = 0 and y1 = 0, and the coordinates of the goal point P2(x2, y2) are recorded after the character “c” in the learned data. With assuming the point P2 to be the climax of the parabolic curve E, the equation E can be denoted as:
y – y2 = m ( xx2 )2.
Furthermore, by substituting the point P1(0, 0) in the above equation, the parameter m can be calculated and shown in the following:
m = y2 / x22.
Finally, the equation E is shown as follows:
E: y = (y2 / x22) ( xx2 )2 + y2. Although the equation E is found out, there is still a problem in the navigation process. That is, the mini-vehicle cannot run along a curvature.
Hence, the mini-vehicle has to navigate along a path which approximates the curve by piecewise linear segments. To find the piecewise linear path, some points Ci of the parabolic curve E are selected before navigation and the mini-vehicle will navigate along these straight segments formed by the selected points Ci. The principle of cutting the parabolic curve is to find the cutting points Ci along the Y-coordinates. A line L: y = a is created to calculate the cross point of the line L and the parabolic curve E, where a = 0 ~ y2. Besides, a threshold value Th is created for use as the condition of selecting cutting points.
At first, the line L: y = a is used to scan the parabolic curve E, where a = 1. The number a will increase until a point C1 is found. As illustrated in Fig. 6(a), a point C1 is selected if θ1
2.50 and θ1 mod 2.5 Th, where θ1 is the angle between the equation of C1P1 and the y-axis and Th is chosen to be 0.1 degrees in our experiment. By increasing the value a, as shown in Fig. 6(b), a general case is that the point Ci+2 is selected if the difference angle θ between the angle θi and the angle θi+1 is larger than 2.5 degrees and θ mod 2.5 Th, where θi is the angle between the equation of CiCi+1 and the y-axis and θi+1 is the angle between the equation of Ci+1Ci+2 and the y-axis. The points Ci and Ci+1 have been selected previously. Finally, the point P2 will be selected and the parabolic curve is segmented by the point set C = {P1, C1, C2, …, Cn, P2}. The navigation path for turning through the corner is formed by the group of segments S =
{
P1C1,C1C2,C2C3,...,Cn−1Cn,CnP2}
.Fig. 5 Procedure of line following.
(a) (b)
Fig. 6 The method of segmenting the parabolic curve. (a) Select the first point C1. (b) The method to select the point Ci+2 in general case.
4.3 Navigation Strategy for Turning Right at Corners
When the learned data are read and two connective actions of the learned data, which are the character “h” followed by the character “r”, are received, the mini-vehicle is commanded to go ahead by line following and dead reckoning and then turn right at the next corner.
The right baseline will disappear before the mini-vehicle arrives at the end of the right baseline because of the limitation of the visual angle of the web camera. Therefore, additional learned data are utilized for the mini-vehicle to navigate continuously. In the learning stage, the distance d of the straight section, which is marked by “h”, is recorded, which is the sum of the length of the right wall and the safe distance mentioned. Accordingly, the strategy of turning right with the hybrid method is proposed as follows. At first, the mini-vehicle navigates forward with the line following method until the right baseline almost disappears, and subtracts the navigation distance d’ from d. Then, the mini-vehicle moves forward for the distance d d’ with the dead reckoning method to reach the turning point. Finally, The mini-vehicle turns right with the angle recorded in the learned data.
By this strategy, the mini-vehicle has the capability to turn right with the learned data, including characters “h”, “r”, the navigation distance and the turning angle.
4.4 Navigation Strategy for Ends of Corridor Sections
The strategy for vehicle navigation at ends of corridor sections is proposed in this section for the mini-vehicle to cross the corridor. When crossing a corridor, the mini-vehicle faces a wall and there are no features in the captured image.
Therefore, a strategy to navigate with the dead reckoning method is designed for this situation
and described as follows.
If the mini-vehicle scans the learned data in the navigation stage and a character “d” followed by a number, say k, is received, the mini-vehicle is ordered to move forward and straightly for the distance k without capturing images. In other words, this strategy is useful in the environment without features because the only information is the learned distance.
5. Experimental results
The images, which are grabbed when the mini-vehicle navigates along the learned path, are shown in Fig. 7. In the following, we first show several representative images during the mini-vehicle’s navigation in a straight section, then the images in a turning section, followed by the images of crossing the corridor, ended with the images of returning to the start point. The navigation speed of the mini-vehicle used in this study is set to be about 10 cm/sec.
Fig. 7 The images grabbed when the mini-vehicle navigates along the learned path.
6. Conclusions
Several techniques have been integrated to design off-line learning and flexible navigation
schemes for autonomous vehicle guidance in general corridor environments. These schemes have been implemented on a test bed, a mini-vehicle, and satisfactory navigation results have been obtained.
A convenient off-line learning technique is proposed by which users only have to know the function keys and the control rules to control the mini-vehicle to learn the environment. This learning technique provides useful data for navigation by recording the user’s commands, gathering environmental features, and processing the learned data in the learning stage.
In addition, a back-and-forth navigation method has been proposed in this study. Three navigation strategies have been proposed in the method to guide the vehicle in different corridor sections. Successful navigations show the practicality of the proposed guidance methods.
REFERENCES
[1] A. M. Martinez, and J. Vitria, “Clustering in image space for place recognition and visual annotations for human-robot interaction,” IEEE Transactions on Systems, Man and Cybernetics, Vol. 31, Issue 5, Oct.
2001, pp. 669-682.A. J. Davison and D. W.
Murray, “Simultaneous localization and map-building using active vision,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 7, Jul 2002, pp. 865-880.
[3] J. B. Hayet, F. Lerasle and M. Devy, “A visual landmark framework for indoor mobile robot navigation,” IEEE International Conference on Robotics and Automation, Vol. 4, 2002, pp. 3942-3947.
[4] Long-Ji Lin, Thomas R. Hancock and J.
Stephen Judd, “A robust landmark-based system for vehicle location using low-bandwidth vision,” Robotics and Autonomous Systems, Vol. 25, Issue 1-2, October 31, 1998, pp. 19-32.
[5] K. Onoguchi et al., “A visual navigation system using a multi-information local
map”, Proceedings of 1990 IEEE International Conference on Robotics and Automation, Cincinnati, OH, U. S. A., Vol.
2, May 13-18, 1990, pp. 767-774.
[6] H. Y. Chiu and W. H. Tsai, “Automatic Vehicle Navigation and Parking in Building Corridors Using Panoramic Sensing and 2D Image Analysis Techniques,” Proceedings of 2002 Conference on Computer Vision, Graphics and Image Processing, Hsinchu, Taiwan, Republic of China, 2002.
[7] G. N. Desouza and A. C. Kak, “Vision for mobile robot navigation: a survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 2, Feb.
2002, pp. 237-267.
[8] K. S. Roh, W. H. Lee and I. S. Kweon,
“Obstacle detection and self-localization without camera calibration using projective invariants,” Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 2, France, 1997, pp. 1030-1035.
[9] G. Y. Chen and W. H. Tsai, “A new approach to vision-based unsupervised learning of unexplored indoor environment for autonomous land vehicle navigation,”
Robotics and Computer-Integrated Manufacturing, Vol. 15, Issue 5, October, 1999, pp. 353-364.
[10] K. P. Li and W. H. Tsai, “Autonomous land vehicle guidance in complex room environments by computer vision techniques”, Proceedings of 1996 International Conference on Image Processing and Character Recognition, Kaohsiung, Taiwan, R.O.C., Dec. 19-21, 1996, pp. 9-16.
[11] Guan-Yu Chen and W. H. Tsai, “An incremental-learning-by-navigation
approach to vision-based autonomous land vehicle guidance in indoor environments using vertical line information and multi-weighted generalized Hough transform technique,” IEEE Transactions on System, Man and Cybernetics-Part B:
Cybernetics, Vol. 28, No. 5, October 1998, pp. 740-748.
