Chapter 4 Obstacle Avoidance by Goal-Directed
4.3 Minimum-Path Following Process
4.3.2 Computation of Goal-Direction Minimum Path
After an obstacle is confirmed to affect navigation, a new path is planned to guide the vehicle to avoid the obstacle. In this study, because our guidance principle is moving sequentially from one node to another according to the navigation map, we
insert a new node before the vehicle reaches the next node, the vehicle will change its navigation path toward the new node. The result is that the vehicle will move to the new node first and go to the original node in the next step.
Under this guidance principle, we can insert a node into the navigation map to guide the vehicle to avoid an obstacle. The key point is how to decide such a node that achieves the purpose. And this node will not guide the vehicle move too far from the next node. Thus, we propose a method to compute a node for goal-direction minimum-path guidance in Algorithm 4.4.
The idea is that we deal with an obstacle once. At first, we transform the obstacle in the image coordinate system to the vehicle coordinate system, so we know the relationship of the obstacle and the vehicle. The computation of a goal-direction minimum path is illustrated in Figure 4.10, where WV is the appropriate width for the vehicle to pass and DG is the shortest distance between the obstacle and the vehicle.
o
Figure 4.10 An illustration of computation of goal-direction minimum path.
We know the front of the obstacle in the vehicle coordinate system, so we can find a point OP of the obstacle which is closest to the vehicle. We also know the left and the right end sides of the obstacle in the vehicle coordinate system and denote them as OL and OR. Thus we can obtain two obstacle avoidance nodes in the vehicle coordinate system in the following:
L' v L P P vehicle coordinate system, and VP is the vehicle position.
Now, we transform OL' and OR' from the vehicle coordinate system to the world coordinate system by Equations 4.1 and 4.2. Then we can obtain the left and the right end points of the obstacle in the world coordinates. We denote the left point as OWL
and the right one as OWR. Thus, the length of each path can be computed in the following way:
L W WL WL next
Length =V OJJJJJJJG + O NJJJJJJJJJG
; (4.5)
R W WR WR next
Length =V OJJJJJJJG + O NJJJJJJJJJG
, (4.6)
where LengthL is the length of the path after inserting the node OWL, LengthR is the length of the path after inserting the node OWR, VW is the vehicle location in the GCS, and Nnext is the position of the next node.
We compare LengthL and LengthR to decide which node should be used for the goal-direction minimum path. We also set a search area which is a semicircle with the radius being the length between the vehicle location and the next node. This search area can guarantee that the new node will not guide the vehicle too far from the next
node by limiting the node in the search area. We use the following formula:
If V NJJJJJGW >r
then N =uN⋅ , r (4.7)
where N is the node for obstacle avoidance, r is the distance between the vehicle and the next node, and uN is the unit vector of V NW
JJJJJG .
In the next cycle, we perform the whole process again to detect and avoid an obstacle. Thus the vehicle can avoid multi-obstacles continuously. Figure 4.11 shows the top view of the obstacle avoidance process.
(a) (b) Figure 4.11 Top view of an obstacle avoidance experiment.
Algorithm 4.4 Computation of goal-direction minimum path.
Input: The obstacle distribution of the front of the obstacle in the VCS.
Output: A node of goal-direction minimum path.
Steps:
Step 1. Compute the two candidate nodes OL' and OR' for obstacle avoidance in the VCS by Equations 4.3 and 4.4.
Step 2. Compute the two lengths LengthL and LengthR for obstacle avoidance with OL' and OR' by Equations 4.5 and 4.6
Step 3. If LengthL is smaller than LengthR, we choose OL' as the obstacle
avoidance node; else, we choose OR'.
Step 4. Using Equation 4.7 to check whether the node is in the search area or not.
If it is not, compute the node position by Equation 4.7.
Step 5. Return the node position.
Chapter 5
Vision-Based Indoor Danger Condition Monitoring
5.1 Principle of Danger Condition Monitoring
One important issue in a vehicle security surveillance system is danger condition monitoring. The earlier danger condition is detected, the less damage is made. It also prevents the danger condition from getting worse. Therefore, to detect danger conditions quickly and correctly is a main issue in this study. However, to detect quickly and correctly is not always a good policy. If the detection is more precise, the computation complication will increase. Once the computation complication increases, the spent time will increase, too. In real time applications, it may lose the best chance easily in time to take some remediable actions.
In Section 5.2, we will discuss the difference between the RGB color model and the HSI color model. The reason for using the HSI color model instead of the RGB color model is also described. In Section 5.3, we discuss the dangerous fire condition and the method we propose to detect a fire. In Section 5.4, we discuss the lighting failure condition, and the method we propose to detect lighting failure.
5.2 Advantage of Using HSI Color Model
The RGB color model presents each color in its primary spectral components of red, green, and blue. A 24-bit RGB color cube is shown in Figure 5.1. A particular color is represented by the amount of each of the primary components. The RGB model simplifies the design of computer graphics systems and is used for color monitors and most video cameras. But it is not ideal for all applications, because the red, green, and blue color components are highly correlated. This makes it difficult to execute some image processing algorithms. Many processing techniques work better on the intensity component of an image. These processes are used more easily and work more efficiently in the HSI color model.
(a) (b) Figure 5.1 RGB 24-bit color cube. (a) Cube with black corner. (c) Cube with white
corner [13].
The HSI color model is one kind of color models describing a color object by its hue, saturation, and intensity. This color model is important and attractive because it represents colors similarly to the way the human eye senses colors. Hue is a color attribute that describes a pure color (pure yellow, orange, or red), whereas saturation gives a measure of a degree to which a pure color is diluted by white light. Intensity is a subjective descriptor [13]. Figure 5.2 shows the HSI color model based on color circles. The hue component contains the color information of the R, G, and B components. The HSI color model decouples the intensity component from the color-carrying information (hue and saturation) in a color image. So the HSI color model allows independent control over hue, saturation, and intensity.
Figure 5.2 The HSI color model based on circular color planes. The circles are perpendicular to the vertical intensity axis [13].
As mentioned before, the RGB color model is ideal for a color camera. The camera used in this study also acquires images in the RGB color model. So we need
to transform the RGB color model into the HSI color model by the following
5.3 Dangerous Fire Condition
Fire is one of the most threatening danger conditions in the world. Detecting fire earlier reduces the damage. Most existing methods for fire detection are based on uses of infrared and ultraviolet sensors. A drawback is that sensors must be installed near the place where fire might be caused, or it will not be effective to detect the fire.
Another drawback is that it can monitor only the place where sensors are installed. In this study, we use a movable vehicle equipped with a camera to perform vision-based fire detection. The vehicle can patrol everywhere in indoor environments to prevent harm caused by fire. The characteristics of fire images are described in Section 5.3.1, and the method proposed for fire detection is described in Section 5.3.2.
5.3.1 Fire Image Features
One difficulty in detecting fire is that fire does not possess a specific shape. It leads to the fact that detecting fire by pattern is hard to carry out. Fortunately, fire
images usually present yellow and red colors. High brightness is also an obvious and important feature of a fire image. We may analyze a lot of images with fire, and check the distribution and histogram of the hue, saturation, and intensity features. The red and yellow colors of fire in images are usually found within some ranges of saturation.
We denote the ranges as Hf1 and Hf2 which are from 0 degree to 45 degrees and from 246 degrees to 338 degrees, respectively. The value of normalized saturation values of fire is larger than 0.1, which we denote as Sf. The normalized value of the intensity of fire in images is almost 1 and we denote it as If.
5.3.2 Proposed Fire Detection Method
While the vehicle patrols in indoor environments, the camera captures images continuously. Each image provides the situation at the time image taking. We analyze the image to judge whether fire exits or not.
In order to detect fire, we need the hue, saturation, and intensity information in the image. We transform the image from the RGB color model to the HSI color model.
The HSI color model is convenient for us to retrieve the information about hue, saturation, and intensity. Then, we filter the images by certain thresholds of hue, saturation, and intensity. On the other hand, we use Hf1, Hf2, Sf, and If to keep necessary components in the image. After the process, we can get three filtered images which fit in with fire. We overlap the three images and keep the intersection to guarantee that the remainders represent fire. Finally, we count the non-black pixels in the result. If the number of such pixels is great enough, it means that a fire is detected.
The complete fire detection process is described in following algorithm and a detailed flowchart is illustrated in Figure 5.3. The result of an image without fire is shown in Figure 5.4. On the contrary, the result of an image with fire is shown in Figure 5.5.
Figure 5.3 Flowchart of fire detection.
Algorithm 5.1 Fire detection.
Input: A color image I.
Output: A fire warning message or nothing Steps:
Step 1. Transform the image I in the RGB color model to the HSI color model by Equation 5.1. The hue is represented by IH , the saturation by IS, and the intensity by II .
Step 2. Keep the pixels in image IH, IS, and II according to the thresholds Hf1, Hf2, Sf, and If in the following way.
2.1 Keep the pixels in the image IH which are among Hf1 and Hf2 Denote the reserved ones as IH’.
2.2 Keep the pixels in the image IS which are greater than Sf. Denote the reserved ones as IS’.
2.3 Keep the pixels in the image II which are greater than ThI. Denote the reserved ones as II’.
Step 3. Overlap the images IH’, IS’, and II’ to obtain the intersection components, and denote the result as Iinter.
Step 4. Compute the non-black pixels in Iinter. Denote the result as Pnum. Step 5. If Pnum is greater enough, then show a warning message.
(a)
(b) (c) Figure 5.4 (a) An image without fire. (b) The hue of (a) within the ranges Hf1 or Hf2.
(c) The saturation of (a) over than Sf. (d) The intensity of (a) equals If. (e) The intersection of (b), (c), (d), and (e)
(d) (e) Figure 5.4 (a) An i e without fire. (b) The hue of (a) within the ranges Hf1 or Hf2.
Figure 5.5 (a) An image with fire. (b The hue of (a) within the ranges Hf1 or Hf The saturation of (a) over than Sf. (d) The intensity of (a) equals If. (e) The int tion of (b), (c), (d), and (e).
mag
(c) The saturation of (a) over than Sf. (d) The intensity of (a) equals If. (e) The intersection of (b), (c), (d), and (e).
(a)
(b) (c)
(d) (e)
) 2. (c)
ersec
5.4 Lighting Failure Condition
The lighting failure is also a common danger condition in daily life. In general, lacking of lighting is very serious because many things and applications depend on lighting, including our system. Therefore, lighting failure detection is an important lighting fails in Section 5.4.1. In Section 5.4.2, we describe the method we propose for lighting failure detection.
sity of images represents the change of the lighting in the ent, and vice versa.
era in a normal situation usually turns out to be an ordinary value. On the
ge difference between the previous intensity image and the present one. Figure 5.6 show
function in our system. We describe the situation and features when
5.4.1 Lighting Failure Features
Lighting influences the image captured by the camera. Actually, the intensity of an image represents the lighting situation in the environment. In other words, the change of the inten
environm
The intensity of an image captured by the cam
contrary, the intensity of an image captured at a lighting failure situation usually becomes a lower value. The difference of the intensity is useful to judge the lighting situation. When lighting fails, the intensity will decrease suddenly. This causes a lar
s the two consecutive images of an ordinary situation and Figure 5.7 shows two consecutive images when the lighting fails.
(a) (b) Figure 5.6 The two consecutive images in ordinary situation.
en the lighting fails. (a) The ordinary
5.4.2 Process for Lighting Failure Detection
hen lighting fails, an effective method to detect the change is to monitor the image continuously. A reasonable assumption is that the environment is ordinary at first.
l by Equation 5.1.
(a) (b) Figure 5.7 The two consecutive images wh
image. (b) The image of lighting failure.
Because the intensity of the image represents the environment changes obviously w
Thus, we capture the initial image and transform it into HSI color mode
Then, we compute the histogram of the intensity of the image to find out the largest and the smallest values. These two values are usually the brightest and the darkest part of the image. However, because of the factor of the light and the camera,
reasons, the darkest data usually represent the part where the light does not cover. We discard these two components and compute the remainder. The mean of the remainder values represents the intensity of this image and we denote it as IntensityA.
Then, we capture an image from the camera again in the next time and use the same method to compute the intensity of the image. Thus, the mean of the intensity of the image is computed and denoted as IntensityB, which represents the lighting situation in the environment at this time. The change of the intensity
B
of two consecutive images comes from IntensityA−IntensityB . The lighting of most situations is not always the same everywhere in general, so the intensity of images is also not the same every time. It needs to tolerate some reasonable changes of the intensity of two continuous images.
However, when the lighting fails, tensity of two consecutive images is obviously, so we use a lot of images to obtain an appropriate value. This experimental value is used to judge whether the intensity changes in a tolerable range or not. Once the change of the intensit
the change of the in
y is large enough, it means that the lighting fails in the environment. We also use the experimental result to obtain another threshold value. Whenever the intensity is below the value, it means that the lighting fails. At the end of each cycle, we update the IntensityA to be the new one IntensityB for the next cycle of lighting failure detection. Thus, we can compare the two latest lighting situations in the environment. The complete fire detection process is described in the following algorithm and a detailed flowchart is illustrated in Some experience results are shown in Figure 5.8, Figure 5.9 and Figure 5.10.
B
Figure 5.11
Algorithm 5.2 Lighting failure detection.
Input A color image I.
Output: A warning message or nothing.
Steps
B color model to HSI color model by epresent the intensity as II.
d discard the highest and the lowest values.
Compute the mean of the remaining intensity values and denote the result
Step 4. ent.
:
:
Step 1. Transform the image I in RG Equation 5.1. R
Step 2. Compute the histogram of II an Step 3.
as IntensityA.
Capture a new image of the latest environm
Step 5. Repeat Step 1 to Step 3 to compute the intensity of the latest image and denote the intensity as IntensityB.
Compute the d
B
Step 6. ifference between IntensityA and IntensityB. If the difference essage.
is out of a tolerable range, then show a warning m Step 7. If IntensityB is too low, then show a warning message.
Update the Intensity
B
Step 8. A to be the new IntensityB for the next cycle of lighting failure detection..
(a) (b) Figure 5.8 The continuous images. (a) The initial situation. (b) The ordinary situation
after (a). (c) The image of the lighting failure. (d) The continuous image after (c).
(c) (d) Figure 5.9 The continuous images. (a) The initial situation. (b) The ordinary situation
after (a). (c) The image of the lighting failure. (d) The continuous image after (c).
(a)
(b)
(c)
Figure 5.10 The intensity histograms respective to Figure 5.X. (a) The mean of the intensity is 0.73. (b) The mean of the intensity is 0.73. (c) The mean of the intensity is 0.55. (d) The mean of the intensity is 0.32.
(d)
Figure 5.10 The intensity histograms respective to Figure 5.X. (a) The mean of the intensity is 0.73. (b) The mean of the intensity is 0.73. (c) The mean of the intensity is 0.55. (d) The mean of the intensity is 0.32.
Transform RGB color
Compare Iavg1and Iavg2
Alarm warning message
Update Iavg1by Iavg2
No
Iavg2 is lower enough? YES
Figure 5.11 Flowchart of lighting failure detection.
Chapter 6
Security Monitoring of Paintings on Walls in Indoor Environments by Image Rectification Technique
6.1 Overview
In the past, a vehicle equipped with a fixed camera can only perform security monitoring of objects on the floor. But it is different to use a vehicle equipped with a PTZ camera. By means of the PTZ camera, security monitoring of objects is no longer restricted to those on floor. Thus, we take the advantage to monitor paintings on walls.
Nevertheless, the vehicle still navigates on the floor, and the view of the camera is still from a low position. An image captured from a low position is always skewed due to perspective transformation. It is inconvenient to get correct information from a
skewed image. T et more precise
information of a painting for security monitoring. The problem of rectification is described in Section 6.2. Once we can rectify a skewed image to get some features of the painting, we need a method to detect a painting on the wall automatically in vehicle security surveillance. A method proposed in this study is described in Section 6.3.
herefore, we have to rectify a skewed image to g
6.2 Rectifica on Walls
In this Section, we discuss the reason why an image should be rectif security monitoring of a painting. In the past, we can only per
tion of Images of Paintings
ied for form security monitoring of objects on floor. Nowadays, with a PTZ camera, security monitoring of objects at higher areas like paintings on walls is possible. However, security
floor is different from the same work in the past. We study in Section 3.2.1. And a method to overcome the problem
6.2.1 Difference Between Images of Paintings on
In general, parallel lines on a scene plane are not parallel in the image but instead converge to a finite point, called vanish point. In this condition, we lose the real shape and the boundary ratio data of an object, which is a painting in this study.
monitoring of an object not on describe the problem met in this
is described in Section 3.2.2.
Walls And Objects on Floors
Because the vehicle is small, the view of the camera on the vehicle is from a low position. In general, if we observe objects in the same height as the camera and face the camera, the image grabbed by the camera looks ordinarily. Once the camera pans or tilts an angle, the image does not look like an ordinary one. It is skewed like the example shown in Figure 6.1. This is a kind of distortion, called perspective transformation coming from perspective imaging. For instance, in Figure 6.1, a card with an original shape of a rectangle is no longer with a rectangular shape in the
Because the vehicle is small, the view of the camera on the vehicle is from a low position. In general, if we observe objects in the same height as the camera and face the camera, the image grabbed by the camera looks ordinarily. Once the camera pans or tilts an angle, the image does not look like an ordinary one. It is skewed like the example shown in Figure 6.1. This is a kind of distortion, called perspective transformation coming from perspective imaging. For instance, in Figure 6.1, a card with an original shape of a rectangle is no longer with a rectangular shape in the