Chapter 3 Learning Strategy for Automatic Navigation
3.5 Learning Processes for Creating a Navigation Path
3.5.3 Algorithm for path learning
In this section, we introduce the method we propose to establish a navigation path in the learning process. We define eleven types of navigation nodes in this study, as
to be conducted by the vehicle. We guide the vehicle to learn a navigation path which we select, and some pre-selected landmarks by the utilization of the navigation node are recorded. Also, other parameters, like the environment intensity and landmark segmentation parameter, are recorded in the learned path. When the learning process is terminated, construction of the navigation path is finished. A navigation path consists of a number of the navigation nodes and some relevant parameters. When the navigation process is started, the navigation path can be used for vehicle navigation successfully. The detailed algorithm to implement the learning system is described in the following, and a flowchart of the process for navigation path creation is illustrated in Figure 3.13
Figure 3.12 A learning interface for the trainer to learn the position of the fixed obstacle by clicking mouse on corresponding points in the image region of Mirror B.
Algorithm 3.4 Creation of a navigation path.
Input: Odometer readings of vehicle poses, denoted as (Px, Py, Pth), where Px and Py represent the vehicle location and Pth represents the direction of the vehicle, in the WCS.
Output: A set of navigation nodes denoted as Npath. Steps.
Step 1. Record into Npath the start node Nbegin of Type 0 with the odometer readings (Px, Py, Pth) = (0, 0, 0).
Step 2. Set the navigation mode, and let the vehicle to navigate forward until arriving at a destination node and stop the vehicle.
Step 3. According to the appointed navigation mode, record into Npath the current vehicle pose, denoted as Ncur = (Px, Py, Pth) obtained from the odometer readings in Type 1 or Type 2; and select one of the following seven additional learning tasks.
(1) Learn a tree landmark by the method described in Section 5.4.2 to obtain a tree position Ntree and the related vehicle pose Ncar, and record Ncar in Type 4 and Ntree in Type 5 into Npath.
(2) Learn a corner of the lawn landmark by the method (with the detail in Section 5.4.2), obtain a tree position Ncor and the related vehicle pose Ncar, and record Ncar in Type 4 and Ncor in Type 6 into Npath.
(3) Learn a traffic cone landmark by the method mentioned in Section 3.4.2, obtain a traffic cone position Ntc and the related vehicle pose Ncar, and record Ncar in Type 4 and Ntc in Type 7 into Npath.
(4) Learn a road stop line landmark by the method mentioned in Section 3.4.2, obtain a road stop line position NWL and the related vehicle pose Ncar, and record Ncar in Type 4 and NWL in Type 8 into Npath.
(5) Learn a signboard landmark by the method mentioned in Section 3.4.2, obtain a signboard position NSB and the related vehicle pose Ncar, and
Section 3.5, and record Nobs in Type 10 into Npath.
(7) Learn a curb line calibration node Ncali, where the vehicle can “see” a complete curb line segment without occlusion and will calibrate its pose by the “seen” curb line information in the navigation process, and record Ncali in Type 3 into Npath.
Step 4. Go to Step 2 if the destination is not reached yet, where the destination position is selected by the trainer.
Step 5. Record the terminal node Nend, denoted as (Px, Py, Pth), according to the current odometer readings, in Type 0 into Npath.
Table 3.2 Eleven different types of navigation path nodes.
Type of number Type of node
Type 0 Start / Terminal node
Type 1 Curb-following navigation node
Type 2 Blind navigation node
Type 3 Curb-line calibration node
Type 4 Localization node
Type 5 Tree landmark node
Type 6 Corner of the lawn landmark node
Type 7 Traffic cone landmark node
Type 8 Road stop line landmark node
Type 9 Signboard landmark node
Type 10 Fixed obstacle node
Set navigation Selected additional learning work
Learn a Tree landmark and a
Learn a corner point of lawn landmark and a related
Figure 3.13 The process for navigation path creation.
Chapter 4
Navigation Strategy in Outdoor Environments
4.1 Strategy for Automatic Navigation
After learning the navigation environment, we get the learned environment information including guidance parameters and a navigation path. Then, we can use them to navigate the vehicle automatically in the learned path. But, it is usually difficulty to let the autonomous vehicle “walk” safely in the complicated conditions on sidewalks. In this chapter, we introduce some strategy for automatic safe vehicle navigation. The principles of conducting the navigation work are described in Section 4.2.1. The detailed algorithm for the navigation process is introduced in Section 4.3.
4.1.1 Vehicle localization by alone-path objects
As mentioned in Chapter 2, Because of manufacturing imprecision, the autonomous vehicle usually suffers from incremental mechanic errors during navigation, causing unstable navigation trajectories. To solve the problem, the strategy adopted in this study is to guide the vehicle to constantly localize its position by the learned fixed landmark position. In Chapter 5, the techniques for localizing a landmark along a navigation path will be introduced. Then we can adjust the vehicle posture by changing its orientation and position to modify the odometer.
4.1.2 Dynamic Adjustment learning parameters in
navigation process
Because of the varying lighting condition on the sidewalk, it is not good to always adopt fixed guidance parameters recorded in the learning process to conduct image analysis works. We adopt a “flexible” strategy to accomplish such works in this study, i.e., techniques for dynamic guidance parameter adjustment are designed for use in vehicle navigation. The techniques are based on Chou and Tsai [23]. Also, we use the contour of the signboard to help the vehicle to adjust the learned parameters for signboard detection by principal component analysis (PCA). The method will be introduced in Chapter 6.
4.2 Guidance Technique in Navigation Process
4.2.1 Principle of proposed navigation process
When the vehicle starts navigation, how to arrive at the node recorded in the learning process is an important issue. In this section, we describe the principles [23]
behind the proposed technique for vehicle navigation on the learned path. When the vehicle prepares to start navigation, it retrieves a navigation path and some guidance parameters which were recorded in the learning process. The navigation path consists of plenty of sequential nodes, so the vehicle can be guided to the destination through the nodes sequentially. Because the vehicle has mechanic errors, it usually reaches the next node at an imprecise location. So some principles are proposed for use to guide the vehicle to correct such errors and navigate safely to the desired destination. They are described as follows.
detect farther landmarks or to avoid obstacles. After detecting and localizing the curb line, the vehicle modifies its orientation to maintain a safe distance with respect to the curb on the sidewalk.
(2). The vehicle localizes its position and corrects the odometer along the navigation path after every constant-time interval. According to the landmark information which we learned, the vehicle detects and localizes the landmark by the use of the proposed landmark localization techniques. Finally, we conduct adjustment of the vehicle pose.
(3). The vehicle always keeps navigating safely by avoiding collisions along the path.
After learning the positions of fixed obstacles, the vehicle conducts a specific procedure to dodge the static obstacle. The procedure will be introduced in Section 4.2.3.
(4). When detecting a landmark using techniques such as dynamic thresholding, the vehicle can adjust guidance parameters if necessary.
General speaking, the vehicle usually localizes itself by the odometer readings to conduct node-based navigation. Considering the mechanic errors, we establish two conditions to decide whether the vehicle has arrived at the next node in node-based navigation. The two conditions are introduced in the following.
(1). Condition 1 as shown in Figure 4.1(a), the distance distX between the current vehicle position V and the position of the next node Nodei+1 should be smaller than a threshold thr1.
(2). Condition 2 as shown in Figure 4.1(b), the distance distY between the next node Nodei+1 and the position of the projection of the vehicle on the vector formed by Nodei and Nodei+1 should be smaller than a threshold thr2.
By checking the above two conditions, the vehicle can be guided to navigate to
the next node and move to the destination. A flowchart illustrating the proposed node-based navigation is shown in Figure 4.2.
Nodei Nodei+1
thr1
V
distX Vehicle current position
(a)
Nodei Nodei+1
thr2
V2
distY
Vehicle current position
(b)
Figure 4.1 Two conditions to decide if the vehicle arrives at the next node in the navigation process.
(a) According to the distance between the vehicle position and the next node position. (b) According to the distance between the next node position and the position of the projection of the vehicle on the vector connecting the current node and the next node.
4.2.2 Automatic vehicle localization by selected landmarks on sidewalks
Although the odometer provides three values Px, Py, and Pth for the vehicle to identify its position (Px, Py) and moving direction Pth, they are in general too imprecise to guide the vehicle to the next position correctly in the node-based navigation. To reduce the influence of the incremental mechanic errors on the vehicle’s navigation, we may employ as many landmarks along the path as possible.
Then, we can use the landmark position to calibrate the odometer reading. In this
shown in Figure 4.3, at first we use the curb line landmark to modify the orientation
(A) Adjustment of the odometer reading of the vehicle position conducted near the curb line.
Figure 4.4 illustrates the relation between two different positions of the vehicle, the curb, and the landmark. The process of adjustment of the vehicle pose using the sidewalk curb line is divided into two steps. When the vehicle arrives at a position which we recorded, we detect the straight curb line segment which is seen in the
Figure 4.2 Proposed node-based navigation process.
omni-image, and calculate the slope angle with respect to the vehicle. Compared with the learned navigation path, we can estimate the current direction of the vehicle.
After adjusting the direction, then we start to detect the landmark and obtain its position. According to the landmark position we recorded in the learning process, we utilize the vehicle orientation which we just calibrate to compute the current vehicle position by the relation between the landmark position and the vehicle position in the GCS as shown in Figure 4.5. The adopted method to calibrate the odometer is described in the following algorithm.
Algorithm 4.3 Conducting adjustment of the odometer reading of the vehicle position near the curb line.
Input: a recorded landmark position Lrecord, the odometer readings of the vehicle pose, a recorded slope angle θ of the curb line, and a recoded vehicle pose VL (Px, PY, Pth).
Output: none.
Steps.
Step 1. Turn the vehicle to the recorded direction Pth, compute the curb line detection process which is described in Chapter 6, and compute the slope angle θ of the curb line relative to the vehicle direction.
Step 2. Compute an adjustment angle θadj by the following equation:
adj = ′ – (4.1)
and modify the orientation odometer reading to be θadj which is then taken as the correct vehicle orientation Pth′.
Step 3. Detect the landmark in the acquired image and compute its position at Lccs in
transformation between the CCS and the VCS described in Equation (3.6) with Lccs in the CCS as input, compute the landmark position LVCS and describe it with coordinates (lx, ly) in the VCS.
Step 4. From the learned navigation path, obtain the recorded landmark position Lrecord
at coordinates (Cx, Cy) in the GCS, and use the calibrated orientation Pth′ to compute the current vehicle position (Xcali, Ycali) in the GCS by the following equations: calculated vehicle position (Xcali, Ycali).
(B) Adjustment of the odometer reading of the vehicle position conducted far off the curb line.
The process for adjustment of the odometer reading of the vehicle position conducted far off the curb line is similar to the above-mentioned process for adjustment near the curb line. The detail of the process is shown in Figure 4.6. First, we detect and localize a nearby curb line segment for the purpose to adjust the orientation reading in a similar way as described previously at a node P1 in the learned path. Next, we conduct a slight difference step, i.e., we guide the vehicle a step further to another node V2, which is a location recorded in the navigation path far off the curb line, for the purpose of detecting the landmark at a close location.
Comparing the two adjustment processes, the second one might cause some mechanical errors and provides erroneous odometer readings after the vehicle moves from V1 to V2. For this, after detecting the landmark and localizing it, we use the same technique to compute the current vehicle position by the relation between the
landmark position and the vehicle position in the GCS as described previously.
Figure 4.3 Proposed odometer reading adjustment process.
θ
Figure 4.4 A recorded vehicle position V and the current vehicle position V in the GCS.
VCS
Figure 4.5 Landmark detection for vehicle localization at position T. (a) At coordinates (lx, ly) in VCS.
(b) At coordinates (Cx, Cy) in GCS.
4.2.3 Fixed-obstacle avoidance process
As mentioned in Chapter 3, there might be fixed obstacles along the path. A fixed obstacle might block the vehicle or cause it to fall outside the sidewalk. So we propose a strategy to help the vehicle avoid the obstacle safely, as described in the following.
As shown in Figure 4.7, the vehicle is navigating to reach a node V1 in the learned path. The next node V2 is just occupied by an obstacle. According to the obstacle parameters Ox and Oy recorded in the learned navigation path which specify how far the vehicle should keep away from the obstacle, we utilize the two parameters to compute three locations by which we can insert three respective nodes in the navigation path as shown in Figure 4.7. The first is placed at the left rear side with respect to the obstacle position. The second is placed at the left front side. And the last is placed right in front of the obstacle. From the third newly-placed node, the vehicle can go back to the original path. By visiting the three new nodes in sequence, the vehicle can dodge the obstacle successfully and be guided to visit node V3.
4.3 Detailed Algorithm of Navigation Process
In this section, we introduce the detailed algorithm proposed in this study for vehicle navigation in the navigation process. With the learned information, the vehicle navigates along the learned path by the way of visiting each recorded node consecutively and finishes works at specific positions until reaching the destination point. The flowchart of the whole navigation process is shown in Figure 4.8. The whole navigation process is described in the following algorithm.
V1(PX1, PY1, PTh1) V2(PX2, PY2, PTh2)
VX
VY
VY
VX
GCS
Vehicle position of orientation reading
calibration Vehicle position of position
readings calibration Landmark
Figure 4.6 Process of odometer calibration position is far off the curb line. The vehicle detects the curb line at V1 to calibrate the orientation and then navigates to V2 to calibrate the position using landmark.
Algorithm 4.3 Navigation Process.
Input: a learned navigation path Npath with relevant guidance parameters, and learned Fixed Obstacle Nodeavoid3.for obstacle avoidance in the original navigation path.
Output: none.
Steps.
Step 1. Read from Npath a navigation node Nnext and relevant guidance parameters.
Step 2. Rotate the vehicle toward the next node Nnext..
Step 3. If a curb line following mode is adopted, modify the vehicle direction after localizing the curb landmark using the dynamic threshold adjustment technique as described in Section 6.1.1 [23].
Step 4. Check if the next node Nnext is reached by the mentioned two principles in Section 4.2.1: if not, go to Step 3; otherwise, continue.
Step 5. If a fixed obstacle node is read from Npath, insert obstacle avoidance nodes into the navigation path and go to Step 8.
Step 6. If a landmark node is read from Npath, take the following steps and then go to Step 8.
(1) Adjust the exposure value to the desired illumination in the relevant environment windows in the image.
(2) Detect the landmark and calculate the landmark position as described in Chapters 5 and 6.
(3) Utilize the position of the landmark to localize the vehicle position and modify the odometer reading as described in Section 4.2.
Step 7. If a curb line calibration node is read from Npath, modify the orientation reading of the odometer after detecting and localizing a curb line segment, as described in Section 4.2.
Step 8. Repeat Steps 1 through 7 until there exists no remaining node in Npath.
Read a new node
Figure 4.8 Flowchart of detailed proposed navigation process.
Chapter 5
Natural Landmark Detection in Images Using New Space Line Detection Technique
5.1 Idea of Proposed Space Line Detection Technique
In this study, it is desired to develop a space line detection method to localize landmarks on the navigation path for vehicle navigation. However, compared to the result of using the traditional projective camera, the projection of a space line on an omni-image using an omni-camera is not a line shape but a conic-section curve [25].
Wu and Tsai [25] proposed a line detection method which detects lines in an H-shaped landmark for use in automatic helicopter landing, as illustrated in Figure 5.1. Firstly, they proved that the projection of a space line in the omni-image is a conic-section curve. Then, by using a 2D Hough transform, they extracted the conic section curve in the omni-image and localized the boundary lines of the H shape for conducting the helicopter landing.
However, Wu and Tsai’s method is based on the fact that the parameters of the hyperboloidal mirror are known. It is known that the parameters of a hyperboloidal mirror cannot be calibrated easily. As mentioned in Chapter 3, it is more convenient for us to utilize the pano-mapping table to calibrate the camera. Thus, we propose new space line detection techniques in this study by the use of the pano-mapping table. We utilize the space plane which goes through the space line and the center of the mirror, instead of trying to obtain directly the conic section curve in the
omni-image. By the use of the two-mirror camera system, the proposed line detection technique is introduced in Section 5.2.1. Besides, we derive a method to obtain 3D information of three vertical lines based on the method for line detection proposed in this study. The derivation of 3D information is introduced in Section 5.2.2.
Finally, by the use of the proposed space line detection technique, many types of landmarks can be detected and utilized for vehicle navigation. We introduce the proposed tree trunk detection method in Section 5.4 and the proposed lawn corner detection technique in Section 5.5. Artificial landmark localization also utilizes these methods, which will be introduced in Chapter 6.
5.2 Proposed Technique for Space Line Detection
omni - camera
(a) (b)
Figure 5.1 Wu and Tsai [25] proposed a line detection method for the omni-image to conduct automatic helicopter landing. (a) Illustration of automatic helicopter landing on a helipad with a circled H shape. (b) An omni-image of a simulated helipad.
5.2.1 Line detection using pano-mapping table
If we want to detect a space line which is projected onto the omni-image, it costs much time to calibrate the camera parameters. In this section, a space line detection technique [23] for use on omni-images which are taken by the two-mirror omni-camera used in this study is proposed. Instead of directly obtaining the conic section curve, we detect the space plane which goes through a specified line and the mirror center. In the following, we describe the proposed detection process.
Assume that a pano-mapping table has been established in advance. Also, assume that the space line L to be detected is projected by Mirror B onto the omni-image, and that G is any space point on L. At first, we propose a way to
Assume that a pano-mapping table has been established in advance. Also, assume that the space line L to be detected is projected by Mirror B onto the omni-image, and that G is any space point on L. At first, we propose a way to