Chapter 6 Collision Avoidance between Vehicles
6.3 Proposed Collision Avoidance Techniques
6.3.2 Collision avoidance on non-intersecting paths
In this state, both paths of the two vehicles must be changed. Each passing point is on the perpendicular bisector l of the path from a current position V = (x1, y1) to a goal G = (x2, y2). Because it is desired to draw apart the distance between the two vehicles quickly, the included angle between the paths V Æ G and V Æ turning point P = (p1, p2) is set by a parameter θ. Therefore, the distance between P and C, which is the midpoint of line segment V and G, is:
2 tan
VG∗ θ. (6.8)
A graphic illustration is shown in Figure 6.5.
V G
The directional vector wJK
of l is perpendicular to the one of path V Æ G, so it is:
Then, l can be represented as:
1 2 1 2
90
Because P is on the perpendicular bisector l and the distance between C and P is computed by Eq. (6.8), P can be represented by Eq. (6.11) below, in which a and b are in the different sides of path V Æ G, respectively:
1 2 1 2
For each path of the vehicles, there are two candidates to choose as a passing point.
The condition of choosing passing points for the vehicles is that the distance between chosen passing points is the longest within all combinations of the candidates. See an example shown in the following.
perpendicular
Figure 6.6Alternative paths at a non-intersecting state.
Points a1 and b1 are the candidates for the first vehicle and points a2 and b2 are the ones for the second vehicle. All states of choosing candidates are (a1, a2), (a1, b2), (b1, a2), and (b1, b2), in which the distance between a1 and a2 is the shortest. So, a1 and a2 are the passing points for the first and the second vehicles, respectively; the alternative paths for the two vehicles are V1 Æ a1 Æ G1 and V2 Æ a2 Æ G2.
Additionally, because the included angle between the two paths from the current
91
position V to the goal G and to the passing point P is restricted by a parameter θ, the distance between P and path V Æ G may be too long. Therefore, we set a fixed distance D between them. If the distance between P and path V Æ G is larger than D, we translate the line segment l connecting V and G along a perpendicular direction toward the passing point P, such as shown in Figure 6.7, in which l is exactly the translation line.
Figure 6.7The passing point P is too far.
The translation line l can be represented as:
2 2
Therefore, the new alternative path is V ÆP1 Æ P2 ÆG. Furthermore, it is necessary to check whether the path is feasible by Algorithm 6.1, and then the final alternative path can be obtained. See an example shown in Figure 6.8 through Figure 6.10 for illustration.
In the example, two vehicles are too close and paths of them are non-intersecting.
Here, we only see the state of one vehicle, as shown in Figure 6.8. P is the passing
92
point, but the point is too far from line segmentVG. Therefore, we translate VG a distance D toward P, and then the translation line l can be obtained, as shown in Figure 6.9.
V
P
G
Rectangular Region R
perpendicular bisector l
Figure 6.8 A passing point P at a non-intersecting state.
V
P
1 G P
P2
D
translation line l
Rectangular Region R
Figure 6.9 P is too far.
After calculating two intersections of l and V Æ P Æ G, the new alternative path is V Æ P1 Æ P2 Æ G. For P1 and P2, it is necessary to check whether they are feasible by Algorithm 6.1. Because the rectangular region which P2 belongs to is different from the one R which both the current position V and the goal G belong to, as shown in Figure 6.10.
93 V
1 G P
P2
θ
P21 P22
Figure 6.10 P2 is not in the rectangular region R.
Hence, we calculate the intersections of P1 Æ P2 Æ G and R, and then the alternative path becomes V Æ P1 Æ P21 Æ P22 Æ G. The processing of computing a alternative path at a non-intersecting state is shown as Algorithm 6.3 below.
Algorithm 6.3 Computing an alternative path for a non-intersecting case.
Input: The current position V1 = (x11, y11) and the goal G1 = (x12, y12) of the first vehicle, the current position V2 = (x21, y21) and the goal G2 = (x22, y22) of the second vehicle, the restricted distance D between a passing point and the original path V1 Æ G1 or V2 Æ G2, and the included angle θ between the original path and the path from the current position to the passing point.
Output: The alternative paths for the two vehicles.
Steps:
Step 1. Calculate two candidates of passing points for the first vehicle (a1 and b1) and the second vehicle (a2 and b2) by the following equation:
94
Step 2. Determine the passing points for the two vehicles in the following way.
Step 2.1. Calculate all distances of the four combinations (a1, a2), (a1, b2), (b1, a2), and (b1, b2).
Step 2.2. Choose the least distance from Step 2.1 and the combination is exactly the passing points for the two vehicles.
Step 2.3. Insert the passing point into the original path for each vehicle.
Step 3. Check whether the distance between the passing point P and the original path is larger than D, for each vehicle.
If yes, calculate the intersections of the translation line l and the alternative path, in which the distance between l and the original path is D; and then, replace P by the intersections.
Step 4. Check whether every alternative path is feasible by Algorithm 6.1.
If not, obtain the new alternative path from Algorithm 6.1.
95
Chapter 7
Experimental Results and Discussions
In this chapter, we show some experimental results of the proposed security patrolling system by two ways. The first is the result of optimal randomized patrolling paths and collision avoidance between vehicles shown by a simulation using programs written in the Borland C++ builder; it is described in Section 7.1. The other is the result of an actual environment in the Computer Vision Laboratory, Department of Computer Science, National Chiao Tung University, and it is described in Section 7.2.
7.1 Experimental Results of Simulation of Patrolling
In this simulation, we create a patrolling environment whose floor shape is composed of four rectangular regions, as shown in Figure 7.1, in which Obj. 0 through Obj.6 are monitoring points.
We utilize the rectangular regions to calculate all the turning points and the distance between each pair of monitoring points, and then save the data into some text files. By reading the files, they can be used again. Furthermore, the threshold to restrict the differences between the patrolling distances needs to be keyed in to the user interface, as shown in Figure 7.1, marked by a red frame. All patrolling paths are random, optimal, and load balanced among vehicles. See an example shown in Figure 7.2, in which the first vehicle starts its navigation at Obj. 4 (M4) and the second
96
vehicle starts at Obj. 1 (M1).
Figure 7.1A simulated patrolling environment.
Among all monitoring points, M0, M3, M4, and M6 are chosen by the first vehicle; M1, M2, and M5 are chosen by the second. The obtained optimal paths are M4 Æ M6 Æ M3 Æ M0 and M1 ÆM2 Æ M5. According to the record of turning points passed by between each pair of monitoring points, obtained in the learning phase, the actual paths are M4 Æ M6 Æ N3 Æ M3 Æ N1 Æ M0 and M1 ÆM2 Æ N2 Æ N3Æ M5 for the two vehicles, as shown by red and green dotted lines in Figure 7.2.
Furthermore, the distances of the paths are 1633.22 and 1081.56. Because the difference of the distances is smaller than the threshold 800, set by the user, the two paths are accepted. In this session, the two vehicles end at M0 and M5, respectively, so the positions are the starts for them in the next patrolling session, for which, the new path planning is shown in Figure 7.3.
97
Figure 7.2 Path planning for the two vehicles in a session.
M0, M2, M3, and M4 are chosen by the first vehicle; M1, M5 and M6 are chosen by the second. The obtained optimal paths are M0 Æ M2 Æ M3 Æ M4 and M5 ÆM6 Æ M1; the actual paths are M0 Æ N1 Æ M2 Æ M3 Æ M4 and M5 ÆM6 Æ N3 Æ N2 Æ M1 for the two vehicles, as shown by the red and green dotted lines in Figure 7.3.
Furthermore, the distances of the paths are 1175.41 and 1262.25 and they are also accepted.
To show the advantage of our system, we compare the times needed for different control factors, as shown in Table 7.1. If the property of randomization is an essential condition, the average time in one session taken by using one vehicle is nearly double of that taken by using two vehicles. This result tells us that the system for multiple
98
vehicles can bring more benefits. Besides, if the number of vehicles is the same, an optimal patrolling path will take less time than a non-optimal path.
Figure 7.3Path planning for the two vehicles in the next session.
For collision avoidance, if the distance between two vehicles is too close, the paths of the vehicles will be changed. The states of non-intersecting paths are shown in Figure 7.4, in which red and green lines are the original paths of the vehicles.
Because the first obtained passing points, red and green circles, are too far from the original path or exceed the walkable regions, the blue circles are calculated. The dotted lines are exactly the feasible alternative paths for the vehicles. Besides, the final passing points belong to the rectangular region which the original path also belongs to.
99
Table 7.1The table of time comparisons where O and X means conducted or not, respectively.
Number of Vehicles
Randomization Optimization
Average Time (second / one session)
Saved Time / Original Time
(%)
1 O O 39.4 -
1 O X 31.6 19.8
2 O O 19.7 50.0
2 O X 13.5 65.7
For collision avoidance of the path-intersecting case, one example is shown in Figure 7.5. Because the second vehicle is closer to the intersection than the first one, the path of the first vehicle must be changed. The black circles a and b are the candidates of passing points. Because b is farther from the goal of the second vehicle, b is chosen.
However, the rectangular region which b belongs to is different from the one which the original path belongs to. The blue circle is the final passing point to be chosen, and the red dotted line is the alternative path for the first vehicle.
100
Figure 7.4Collision avoidance of non-intersecting paths.
101
Figure 7.5 Collision avoidance of intersecting paths.
7.2 Experimental Results of Patrolling in Real Environment
The real environment for this experiment is an open space area in our laboratory.
Because autonomous vehicles used in the study suffer from accumulation of mechanical errors, two top-view omni-cameras are utilized to locate and monitor the vehicles.
While collecting data for Table 7.2, we drive the vehicle to random places and record the values of the actual positions and the odometer. The total moved distance, passing twenty position points, is 4818.40 centimeters and the average error rate without calibration by tow-view cameras between the actual positions and the
102
odometer values is 8.86%. Furthermore, the reason why the errors do not increase is that turning of the vehicle also incurs errors, so the errors might cancel one another by left and right turnings.
Table 7.2Mechanical errors of the vehicle.
(1)Actual Position (2)Odometer Value No.
Additionally, we also record the errors of the two top-view omni-cameras. The image in Figure 7.6 is the view of the first camera and we calculate the positions of all circles in the image.
In Table 7.3, we calculate the errors between the actual positions and the positions in the image. From the values, we know that the errors of those points,
103
which is farther from the view center of the camera, is bigger. However, the average error rate with calibration is only 3.86% and all points are independent. Therefore, the vehicles are located by the cameras such that the vehicles do not suffer from accumulation of mechanical errors anymore. About the second camera, the view and the errors are shown in Figure 7.7 and Table 7.4, in which the average error rate with calibration is 2.51 %.
Figure 7.6 The view of the first top-view omni-camera.
Furthermore, the task of security patrolling includes the work of capturing the pictures of some monitored objects. By the top-view omni-cameras to locate the vehicles periodically in the patrolling session, the vehicles can accomplish the mission with the information of the positions and the orientations with respect to the objects, obtained in the learning phase. In the following, we show some results of images taken by the vehicles. Some monitored objects are in the center of the images, as shown in Figure 7.8(a). The images, captured by the vehicles in the patrolling session, with respect to the ones in Figure 7.8(a) are shown in Figure 7.8(b).
104
Table 7.3Errors of the first top-view omni-camera.
(1)Actual Position (2)Image Position
No. X y x y
In Figure 7.8, the difference between each pair of images is smaller. It tells us that the proposed vehicle-pose learning strategy and the proposed vehicle localization technique are good for the vehicles to perform the security patrolling task.
105
Figure 7.7The view of the second top-view omni-camera.
Table 7.4Errors of the second top-view omni-camera.
Actual Position Image Position
No. x y x y
Error (| (1) (2) |
(1)
− cm)
1 488 30.5 484.88 23.73 0.109 2 549 30.5 542.84 23.49 0.074 3 610 30.5 608.29 24.17 0.035 4 671 30.5 671.32 24.42 0.025 5 732 30.5 736.11 24.35 0.024 6 793 30.5 796.44 23.47 0.021 7 457.5 91.5 456.75 87.34 0.044 8 518.5 91.5 516.22 87.75 0.034 9 579.5 91.5 575.82 86.18 0.036
10 640.5 91.5 637.75 87.95 0.019
11 701.5 91.5 703.29 86.42 0.019
12 762.5 91.5 766.8 85.91 0.020
13 823.5 91.5 824.58 86.1 0.014
14 457.5 152.5 454.96 150.7 0.020
15 518.5 152.5 514.43 152.5 0.023
16 579.5 152.5 574.83 149.27 0.027 17 640.5 152.5 640.64 149.46 0.012 18 701.5 152.5 703.11 150.98 0.007
106
Actual Position Image Position
19 762.5 152.5 764.38 150.99 0.007 20 823.5 152.5 824.02 149.14 0.008 21 427 213.5 429.35 215.86 0.016 22 488 213.5 485.81 213.63 0.010 23 549 213.5 543.7 212.1 0.022 24 610 213.5 606.87 212.05 0.012 25 671 213.5 671.08 211.9 0.005 26 732 213.5 735.21 212.23 0.009
(a) (b)
Figure 7.8The security patrolling task. (a) Images captured in the learning phase. (b) Images captured in the navigation phase.
107
(a) (b)
Figure 7.9The security patrolling task. (a) Images captured in the learning phase. (b) Images captured in the navigation phase. (continued)
108
7.3 Discussions
The proposed system utilizes multiple vision-based autonomous vehicles to perform the security patrolling task. For this purpose, some monitoring points are utilized to guide the vehicles. By the way, there are more applications of the monitoring points, such as providing various services. Every monitoring point can be regarded, for example, as a business service point in which there are some customers.
If the environment is a restaurant, the apparatus of showing menu can be equipped on the vehicles, and then the vehicles can move to each service point along assigned optimal paths to ask what dishes or services are needed. If the environment is a company, the vehicles also can be utilized to deliver documents or messages in each service point. Furthermore, if a walkable area can be divided into many ranges, in which each is within the controllable view of the vehicle, we may transform every range into a node such that the vehicles can arrive at anywhere in it to do some actions, such as detecting whether an unknown person has invaded with optimal randomized paths.
However, there are still some problems in the system. If an object appears next to the vehicle suddenly, the top-view omni-cameras will not have the ability to find out the vehicle. To solve the problem, it might be necessary to add information of color and sample models of the vehicles to this system. Furthermore, the vehicles are not on a plane, so the vehicle localization accuracy is affected by the heights of the vehicles.
If the vehicle is taller and farther from the top-view omni-cameras, the error between the obtained centroid and the actual position of the vehicle will be large. However, we might be able to add an obvious mark on the center of the top of the vehicle. By finding the mark, the correct position can also be obtained. Finally, the proposed real-time collision avoidance technique between vehicles is feasible for two vehicles.
109
If the total number of vehicles is larger than two, we will need to consider the influence of passing points for the third vehicle. The problem is worth for future research.
110
Chapter 8
Conclusions and Suggestions for Future Works
8.1 Conclusions
In this study, we utilize multiple vision-based autonomous vehicles to develop a security patrolling system in an environment whose floor shape is composed of rectangular regions. We have proposed several techniques and adopt some algorithms which are summarized in the following.
(1) An environment-information calculation method has been proposed, by which we can obtain all rectangular regions, which form the floor shape of the patrolling environment, the turning points, and then all between-MP distances and paths.
The turning points are utilized to enable the vehicles to move between any pair of MPs without collisions with the walls. With the turning points, we adopt Dijkstra’s algorithm to obtain the shortest between-MP distances and paths between the two MPs which belong to the different regions.
(2) A point-correspondence technique integrated with an image interpolation method for camera calibration has been proposed. In this study, we don’t use the traditional projection-based transformation. Instead, a grid pattern is used as the calibration target and corresponding points between 2-D image and 3-D global spaces are utilized. For the warped images captured by the top-view omni-cameras, the correct coordinate positions can be obtained by the
111
corresponding points and the use of an image interpolation method.
(3) A faster point-correspondence technique has been proposed. Because more corresponding points will yield better calibration accuracy, we adopt a minimum mean square error (MMSE) method to calculate quadratic curves and abundant cross points, in the image captured by the top-view omni-camera, can be obtained. Each cross point and its coordinates in the global space, obtained by an interpolation method, are exactly one pair of point correspondences.
(4) A vehicle-pose learning method has proposed, by which the vehicles are taught where and in which direction to perform the security monitoring task, which is to take pictures of monitored objects as defined in this study. Furthermore, the learned positions can be utilized to guide the vehicles.
(5) An optimal method for randomized and load-balanced path planning has been proposed, in which each MP is just passed once such that monitored objects can be patrolled uniformly. Additionally, the difference of the numbers of assigned MPs for all vehicles is smaller and a threshold distance is set to restrict the difference between path distances, so that the loads of all vehicles can be balanced. According to the numbers of assigned MPs, the MPs are chosen randomly, and then the system calculate the shortest paths with each MP on these paths appearing only once by the concept of the TSP.
(6) A vehicle localization and monitoring method has been proposed. Because the vehicles suffer from mechanic errors, we utilize the top-view omni-cameras to locate them in this study. By the odometer values of the vehicles, we can calculate the centroids of the vehicles in the image. After the centroids are transformed into the global space, the odometer values are corrected by the coordinates of the resulting points. Besides, the directional angles of the vehicles also must be corrected, in which two continuous correct position points are
112
utilized to do the job. Additionally, the cameras have the ability to monitor vehicles to see whether they are still under control. If any vehicle loses control of its action, the system will send an alarm message to the security center and stop all vehicles.
(7) A real-time collision avoidance technique between two vehicles has been proposed. By the odometer values, the system computes the distance between two vehicles in every cycle of a fixed-time duration and determines whether they are too close. If yes, the feasible alternative paths of the vehicles will be obtained by two different kinds of states, path-intersecting or non-path-intersecting.
The experimental results shown in the previous chapters have revealed the feasibility of the proposed system.
8.2 Suggestions for Future Works
The proposed strategies and methods, as mentioned previously, have been implemented on a vehicle system with multiple vision-based autonomous vehicles.
According to this study, in the following we make several suggestions and point out some related interesting issues, which are worth further investigation in the future:
(1) using a pen-tilt-zoom camera equipped on the vehicle to capture clearer images, and then extracting features of the images to detect whether monitored objects still exist;
(2) adding the capability to detect more danger conditions;
(3) adding the capability of warning users immediately through cell phones or
(3) adding the capability of warning users immediately through cell phones or