Chapter 4 Learning Procedures
4.5 Learning of 2D Objects
4.5.4 Finding the Horizontal Line Automatically
If we want to find the horizontal line, then we need to find the set of edge points by edge detection in the image first. There are many different operators which can find edges in an image, for example, the Laplacian operator, Marr-Hildreth operator and Canny operator, etc. In our system, we detect edges by using the Sobel operator as shown in Figure 4.14. Let T be the threshold value of the Sobel operator.
-1 -2 -1 Figure 4.14 Sobel Operator (a) x direction (b) y direction.
We apply the Sobel operator on a pixel pi, and let rpi be the resulting value. If Inequalities (4.19) below is satisfied, then we mark the pixel pi as an edge point.
rpi > T (4.19)
After checking very pixel pi in the interesting region which was obtained from Section 4.5.3, we can get a set of edge points, EP.
Now we can find the line by using the Hough Transform. It is a method of line detection by a voting technique and the relationship between the parameter space and the normal distance-normal angle space is as shown in Figure 4.15.
Figure 4.15 The relationship with x-y space and γ-θ space.
At first, we divide the range of angles, [0, π], in to n part, Θ0, Θ1, …, Θn, and calculate the length rregion of a diagonal of the interesting region. We prepare next an accumulation array A, whose size is n×rregion, and set the value of every cell to zero, as shown in Figure 4.16.
T0 T1 Tn-1 Tn
r0 r1
rregion
...
...
T r
Figure 4.16 The accumulation array A of the Hough Transform.
We substitute a point (x, y) in the edge point set EP and Θi into Eq. (4.20) below to get a value r and put a vote into the cell (r, Θi):
cos i sin i
r=x θ +y θ . (4.20)
We repeat this step of voting in the cells for every edge point and every Θi. After voting, we find the cell (γ, θ) with the maximum number of votes to obtain a line which has the intercept γ and the angle θ in the normal distance-normal angle space.
There is a restriction in our method, that is, the heights of monitored objects must be of fixed values known in advance. By the progressive method as described above, we can find the interesting region of the concerned 2D object in the image I and the horizontal line L, which is parallel to the floor plane in the 3D global coordinate system in image I, as shown by the example in Figure 4.17 and Figure 4.18.
(a) (b)
Figure 4.17 An example of detection of information of monitored object by the progressive method. (a) An image with a monitored object. (b) The thresholding result. (c) The interesting region. (d) The horizontal line.
(c) (d)
Figure 4.17 An example of detection of information of monitored object by the progressive method. (a) An image with a monitored object. (b) The thresholding result. (c) The interesting region. (d) The horizontal line.
(continued)
(a) (b)
Figure 4.18 Another example of detection of information of monitored object by the progressive method. (a) An image with a monitored object. (b) The thresholding result. (c) The interesting region. (d) The horizontal line.
(c) (d)
Figure 4.18 Another example of detection of information of monitored object by the progressive method. (a) An image with a monitored object. (b) The thresholding result. (c) The interesting region. (d) The horizontal line.
(continued)
Chapter 5
Path Planning by Minimizing Mean Square Errors Using Ultrasonic
Signals
5.1 Introduction
After learning the path and the monitored object data, the system will refine the path data before the vehicle navigates to the start point. The path data are composed of many path nodes. Two extreme examples of path planning results are shown in Figure 5.1(a) and Figure 5.1(b). The First example is a path for backing to the start point directly which is very simple and crude, and there might have some obstacles on the path. The second example is a path stepping on every node precisely. It may not seem smart because the path could be rough or tortuous. Hence, we desire the planned path could be smooth but not lose the original trend, as the illustration shown in Figure 5.1(c). We describe the details of the proposed path planning process in Section 5.2.
The system performs the path planning process by using the MSE criterion. In the MSE criterion, the system needs to set a threshold value which is used to control the precision of the path. Hence, the choice of the threshold value is very important.
Our system uses a technique which can choose the threshold value by adapting to different environments. We describe the detail of the proposed process in Section 5.3.
In Section 5.4, we will describe the details of the patrolling technique of our system.
Start
Figure 5.1 Illustrations of path planning. (a) Backing to the start point directly. (b) Stepping on every path node precisely. (c) The planned path we desire.
5.2 Path Planning by MSE criterion
The path data are composed of path points. As a start, the system finds out n path points where we learned monitored objects, and separates the path into n+1 segments by the n path points. Then we construct an empty queue Q, and push the n+1 segments into Q.
The system pops out a segment of the path, and uses the line fitting technique which is described in Section 3.3.3, to find a linear equation of a line L in two variables, x and y, like the formula of Eq. (5.1) below, where A, B and C are constants:
: 0
The value of the induced square error E of the line L can be calculated by
The system uses a threshold value Tmse as the upper bound of the square error value. If the error E is smaller than Tmse, then the system considers the currently-processed segment of the path smooth enough. But if the error E is larger than Tmse, then the system considers the segment not smooth enough, and we need to do more processing for this path.
It is proposed in this study to choose a cut point from the path points P1, P2, …, Pm of the segment of the path which is not smooth enough. If we cut on the point Pj, then we need to find the line L1j and Ljm by using the line fitting method again and calculate the values of the two mean square errors E1j and Ejm separately. Then, the square error of cutting on the point Pj is
After cutting the path m-2 times on P2, …, Pm-1, we can get m-2 mean square errors E2, E3, …, Em-1. We then find the minimum value Ek from the m-2 error values, with Ek
meaning that if we cut the path on Pk, then we can get the least error value. We finally push the two shorter paths P1, P2, …, Pk and Pk+1, Pk+2, …, Pm into the queue Q.
The system executes the above process repeatedly, until the queue is empty, and at that time the original path has been cut into many segments which are smooth enough. Finally, we save every start point of these segments of the path as the navigation path data. An illustration of the path planning process is shown in Figure 5.2.
1
j j jm
E =E +E . (5.4)
Allowed the part of path
Separate path and push path data into an empty queue
Object
Line fitting and find error value Ej = E1j+ Ejm
j = m-1 ? j=1 j = j +1
Find minimum Ekof E2…Em-1
Figure 5.2 Illustration of proposed path planning.
5.3 Dynamic Path Decomposition
The threshold value of the MSE criterion is used to tolerate the value of the error.
The larger the threshold, the cruder the path decomposed. There is no advantage if the threshold is very large or very small. The choice of the threshold should adapt to the environment of the path. Moreover, the environment could be different at different parts of a path, and each different part of the path should be decomposed by a different degree of precision. We propose a technique here which chooses the threshold value dynamically.
Except the coordinates of path points, the distances are also part of the information of path data which are learned in the learning process by using ultrasonic sensors. Our system knows the breadth of the environment of a path by analyzing the distance data collected by the ultrasonic sensors. In the learning process, the system learned the left and right distances (dl,i, dr,i) of every path point Pi with respect to the obstacle or the wall around the point. We simplify such distance data to get a single distance value Di of every path point by
Considering the correlation of the connected segments of the path, our system calculates the threshold value of one segment by using the distance values of not only the path points in this segment but also five points before and five points behind this segment. Additionally, the composite the mean square error for this segment is taken to be the average of the squares of the error distances, as Eq. (5.3), and the threshold value is related to the mean square error. So we derive the equation of Eq. (5.6) below to calculate the threshold value of a segment with n points;
(
, ,)
min ,
i l i r i
D = d d . (5.5)
where k is a constant.
We decompose the path dynamically by recalculating the above threshold value in every cycle of a navigation session. An illustration of the proposed dynamic path decomposition scheme is shown in Figure 5.3.
Path Planning
If the queue is empty? End of path planning
Pop a segment of path Pi~Pj from queue
Figure 5.3 An illustration of proposed dynamic path decomposition.
1 5
5.4 Vehicle Navigation by 2D Object Image Matching
After planning the path, the last process of the system is vehicle patrolling from the end to the start position of the path obtained from the path planning process. In a patrolling session, the vehicle navigates along a learned path by visiting each path point consecutively. The learned navigation path, which consists of a set Npath of points and we specify the last point of Npath to be the initial position of the navigation path. Then, the vehicle reads the next node data Ni+1(xi+1, yi+1) and computes a turning angle and a moving distance by Eqs. (5.7) and (5.8) in the following algorithm, for the vehicle to move to the next position Ni+1(xi+1, yi+1), as shown in Figure 5.4.
Algorithm 5.1. Process of vehicle guidance by a learned path.
Input: A set Npath of nodes.
Output: The vehicle navigates to the start point in the learning process.
Steps:
Step 1. Start vehicle navigation from the starting node N0 in Npath.
Step 2. Let the current position be Ni(xi, yi), and the direction read from the odometer be θodo.
Step 3. Scan Npath to read the next node Ni+1(xi+1, yi+1).
Step 4. Compute the moving direction as a vector Vi by using the following equation:
Step 5. Compute the direction angle θnew for the vehicle to turn toward the node
Ni+1 by using the following equation:
⎟⎟⎠
⎜⎜ ⎞
⎝
= − ⎛
i i
new X
1 Y
θ tan . (5.8)
Step 6. Compute the navigation distance for the vehicle to advance as d = Vi . Step 7. Calculate the odometer calibration angle θcali using the following equation
obtained from Section 3.3.3:
0.00000476 0.00592048 4.164379512
cali d d
θ = × + × + . (5.9)
Step 8. Compute the rotation angle for the vehicle as θturn = θnew − θodo + θcali. Step 9. Turn the vehicle leftward for the angle θturn if θturn is larger than zero;
otherwise, turn the vehicle rightward for the angle θturn. Step 10. Move the vehicle forward for the distance d.
Step 11. Read the next node data. If there exist remaining nodes, repeat Steps 3 though 8; else, finish the navigation.
Figure 5.4 Computation of the turning angle and move distance.
Additionally, the navigation path is composed of two kinds of special path points.
The first kind is turning point where the vehicle turns a large angle θfind, obtained from Section 4.2.3. Because θfind is a large angle, we hope the vehicle can really turn this angle in the navigation process; otherwise, the resulting vehicle trajectory will be too far away from the desired path learned in the learning stage, according to our experience of experiments conducted in this study. Hence, when the vehicle navigates to this kind of path point, it is commanded to turn to the inverse direction of the angle θfind + θcali instead of the angle θturn calculated in the algorithm above. The second kind of path point is monitoring point where the vehicle learned a 2D object, obtained from Section 4.5.2. When the vehicle navigates to this kind of path point, the vehicle will “see” the 2D object again and the system will monitor this object. After successfully monitoring an object, we can take advantage of the matching result to adjust the vehicle location [9]. An illustration of the patrolling process is shown in Figure 5.6. Some examples of experimental results are shown in Figure 5.5 and Figure 5.7.
Figure 5.5 An example of a planned path of an ellipse shape.
Patrolling
Current point is a turning point ?
End of patrolling
Monitor object and adjust position
Calculate the distance d to the next data
Compute the θcali
Read path data of current point and next point
θturn=θfind+θcali
Move the vehicle forward for the distance d θturn=θnext-θnow+θcali
Turn angle θturn Current point is a monitoring point ?
Arrive start point Yes
No
Yes
No
Yes
Figure 5.6 An illustration of proposed patrolling process.
(a) (b)
(c) (d)
(e)
Figure 5.7 An example of experimental results. (a) The vehicle monitors an object. (b) The vehicle walks to a turning point (c) The vehicle monitors another object. (d) The vehicle goes back to the original start point.
Chapter 6
ALV Navigation by Ultrasonic Signal Sequences
6.1 Ultrasonic sensing in ALV System
The vehicle of our system has eight ultrasonic sensors that facilitate implementations of object detection and range finding functions for collision avoidance, recognition, localization, and navigation. The ultrasonic sensors are equipped in the vehicle at fixed locations around the vehicle: one on each side and six facing outward at twenty degree intervals, as shown in Figure 6.1.
The ultrasonic sensors may be used to obtain the information of distances by calculating the time spent from the start of firing an ultrasonic sound to the end of receiving a reflected sound. The distance data could have errors if the plane, which receives the incident, is not perpendicular to the fired sound ray. If the angle of incidence grows, then the erroneous distance values will grow, too. This phenomenon needs to be noticed when using this kind of sensor.
In our system, we set the left distance DL as the value detected from ultrasonic sensor No. 0 plus half of the width of the vehicle, set the right distance DR as the value detected from ultrasonic sensor No. 7 plus half of the width of the vehicle, and set the front distance DF as the average value of those detected from ultrasonic sensors No. 3 and No. 4. The system catches the reflected values Femit times every second as a sequence of distance data.
Left Right
Figure 6.1 The positions of ultrasonic sensors around the vehicle.
6.2 Principle of Navigation
6.2.1 Learning Strategy
The navigation paths in an indoor environment are usually composed of some straight lines which are connected by turning points. Before navigation, the vehicle has to learn the parameters of Width, Directionturn, Directiondetect, and Distancedetect of the environment. For a straight line of a path in the indoor environment, if it is a hallway with two walls at both the left and the right sides, then the value Width is set equal to the width of the hallway. But if the straight path only has one wall at the left or the right side, then Width is set equal to double the distance between the vehicle and the wall. An illustration is shown in Figure 6.2. For a turning point of the path, if the vehicle needs to turn to the right at the turning point, then we set Directionturn = right, and if the vehicle needs to turn to the left, we set Directionturn = left.
Width
(a)
1 Width×2
(b)
Figure 6.2 An illustration of the parameter of Width. (a) The hallway with two walls.
(b) The hallway only has one wall.
Additionally, the vehicle has to know the conditions of turning points. Turning points are usually located at the end of a wall or a hallway. The distance values caught from the ultrasonic sensors will vary to large values at the position of the end of a wall. The parameter Directiondetect is used to save the direction where the environment has caused such a kind of large-value variation and the vehicle can detect such variations to find out turning points. If a turning point is located at the end of the left wall, then we set Directiondetect = left; otherwise, we set Directiondetect = right.
Moreover, we set Directiondetect = left or right to mean that the turning point is located at the end of the left or the right wall, respectively. If the turning point is located at the end of the hallway, it means that there is something stopping the hallway. In that case, we set Directiondetect = front. The last parameter needs to learn is Distancedetect which is the distance between the turning point and the start position of the end of the wall in the environment. An illustration is shown in Figure 6.3.
6.2.2 Navigation Strategy
After learning the navigation path in the environment, the vehicle can navigate along the learned path by analyzing the sequential signals acquired by the ultrasonic
sensors. The vehicle has two missions, one to navigate along the middle line of the hallway, and the other to keep the direction of navigation parallel to the hallway.
Turning Point Distancedetect
(a)
Turning Point
Distancedetect
(b)
Turning Point Distancedetect
(c)
Turning Point Distancedetect
(d)
Figure 6.3 An illustration of the parameter of Directiondetect (a) Directiondetect = left.
(b) Directiondetect = right. (c) Directiondetect = left + right. (d) Directiondetect
= front.
We begin a navigation session at a start position in the hallway, and let it go forward by a fixed speed Speed. From the ultrasonic sensors, the system retrieves distance information Di which includes the left and the right distances (Di,L, Di,R) at time i, and analyzes n sets of data, Di-n, Di-n+1, …, Di, before time i. The vehicle needs to adjust its direction in four situations. When the vehicle moves close to the wall at the left side gradually, the system could find out the fact that the left distance of the vehicle is reducing or that the right distance of the vehicle is growing. Then, if the vehicle is close enough to the left wall, that is, if the right distance of the vehicle
exceeds half of the width of the hallway, then the vehicle turns an angle θlittle to the right, that is, the vehicle turns when the following conditions are met:
, 1,
On the other hand, when the vehicle moves close to the wall at the right side gradually, the system could find out the fact that the right distance of the vehicle is reducing or that the left distance of the vehicle is growing. Then, if the vehicle is close enough to the right wall, that is, if the left distance of vehicle exceeds half of the width of the hallway, the vehicle turns an angle θlittle to the left, that is, the vehicle does so if
for every k between i − n and i. An illustration of the situation of adjusting the direction of the vehicle is shown in Figure 6.4. An illustration of navigation in a hallway is shown in Figure 6.5.
It is seems superfluously if we both consider Eq. (6.1) and (6.2), but it is necessary. The walls in environments are not completely planar and the sensor could get some noise, so we need to consider the data from both sides of the wall. Eq. (6.4)
and (6.5) are for use to handle the same situation as above.
Figure 6.4 An illustration of the situation of adjusting the direction of the vehicle. (a) The vehicle should turn an angle to the right (b) The vehicle should turn an angle to the left
Left distance > half width of hallway
Figure 6.5 An illustration of navigation in a hallway.
For detection of a turning point, we have learned the parameters of the direction
For detection of a turning point, we have learned the parameters of the direction