In this chapter, we illustrate our proposed scene analysis system by simulating the outdoor images, which are chromatic image with size 256×192. These images are taken on freeway by using digital camera. In order to confirm the validity of the proposed system, we provide the desired output image by manual recognition of segmented natural objects. Then, we compare the simulated results with the desired output image. This simulation was done on Pentium4 2.4G personal computer.
4.1. Fuzzy Rule Base for Freeway Scene
As mentioned in Chapters 2 and 3, we design a scene analysis system based on fuzzy ID3 algorithm. With this algorithm, we can establish a fuzzy rule base to analyze the freeway image taken from a driver view in a driving car.
As developed by Wang and Mendel [20], fuzzy rules were generated by learning from examples. A training data with feature vector and is associated with the desired output of corresponding objects. Such image pixel constitutes an input-output pair.
These fuzzy rules are a series of associations of the form “if antecedent conditions hold, then consequent conditions hold.” For our system, we consider the consequent conditions are the name of the natural objects in the scene such as sky, tree, road, vehicle, and others. And the number of antecedent conditions equals the number of features.
(a) (b)
(c) (d)
(e) (d)
Fig. 4.1. (a) four 128×96 training images, (b) desired output image, (c) gray level of the training images, (d) hue of the training images, and (e) training result by our proposed scene analysis system.
In order to construct the fuzzy ID3 decision tree to specify the fuzzy rule base to analyze the image scene, we use four typical 128×96 outdoor images as our training data. These outdoor images are shown in Fig. 4.1(a). Then these features used for the training data are gray level, hue, and vertical position. Gray level and hue are shown in Figs. 4.1(c) 4.1(d), respectively. As to these images, the following five classes will be assigned: the sky is denoted by white, the tree is denoted by green, the road is denoted by gray, the vehicle is denoted by yellow, and the others is denoted by red.
Fig. 4.1(b) is the desired output image, obtained by manual segmentation.
Subsequently, we use these data as our training data set. After running the genetic algorithm based fuzzy ID3 method, the scene analysis system generates 30 fuzzy rules and provides a training result which is shown in Fig. 4.1(e). After recognizing by this fuzzy rule base, the training data set has obtained an 87.5% accuracy of region segmentation. In the following, we utilize this fuzzy rule base obtained as the default fuzzy rule base in the consequent freeway scene analysis
4.2. Simulation Results
After establishing the fuzzy rule base, we apply our scene analysis system to analyze 20 chromatic images taken from a driving car on freeway by using digital camera. Here, the first stage output represents the output after fuzzy rule base inferring, the second stage output represents the output after image ground-truthing refining, and the third stage output represents the output after image erosion shrinking.
These analyzed results are shown in Figs. 4.2–4.21. In the intelligent transportation system, the vehicle recognition rate is the most important figure to be considered.
Therefore, in addition to the image accuracy of segmented natural objects, we use another criterion AV defined by
to evaluate the vehicle recognition accuracy of our proposed scene analysis system.
Vehicle pixels in the segmented image that is classified correctly – Non-vehicle pixels in the segmented image that is classified to vehicle class
Total vehicle pixels AV =
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.2. Testing on Image 1. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.3. Testing on Image 2. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.4. Testing on image 3. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.5. Testing on Image 4. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.6. Testing on Image 5. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.7. Testing on Image 6. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.8. Testing on Image 7. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.9. Testing on Image 8. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.10. Testing on Image 9. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.11. Testing on Image 10. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.12. Testing on Image 11. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.13. Testing on Image 12. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.14. Testing on Image 13. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.15. Testing on Image 14. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.16. Testing on Image 15. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.17. Testing on Image 16. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.18. Testing on Image 17. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.19. Testing on Image 18. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.20. Testing on Image 19. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 4.21. Testing on Image 20. (a) original image, (b) desired output image, (c) resulting image by fuzzy rule base inferring, (d) possible vehicle region finding by image ground-truthing, (e) vehicle region refining by edge detection, (f) final scene image obtained by image erosion, and (g) lane line recognition.
In summary, the overall accuracy comparison of the above 20 images between each stage output are illustrated in TABLE I. After testing 20 chromatic images, we obtained 87.79% accuracy of the overall scene images and 84.12% accuracy of vehicles in the scene images. As demonstrated in the successful application on freeway images, the proposed scene analysis system is general and robust.
TABLE I
ACCURACY COMPARISON BETWEEN EACH STAGE OUTPUT
Considering all vehicles in the image
Ignoring far away vehicles and specify far away vehicles as others Image accuracy Vehicle accuracy Image accuracy Vehicle accuracy 1st stage output 84.91% 35.57% 84.02% 28.35%
2nd stage output 87.93% 73.08% 88.88% 84.06%
3rd stage output 87.66% 73.87% 87.79% 84.12%