Lane Detection Results

1) Dynamic calibration of camera tilt angle

Figure 3-14 demonstrates the result of the dynamic calibration of camera’s tilt angle. In the figure, ”Original” means the calculated tilt angle in each frame. ”Kalman” denotes the processed tilt angle by a Kalman filter. The Kalman filter can provide the robust estimation of the current tilt angles through recursive functions [91][92]. This process provides the more stable and robust calibration results of the tilt angle for the lane detection system. Figure 3-14 shows that the change of ”Kalman” gets smaller.

Fig. 3-14. The result of the dynamic calibration of camera’s tilt angle.

Conditions The number of

Fig. 3-15. The estimated lane width in every frame.

(a)

(b)

(c)

(d)

Fig. 3-16. The gray level of lane markings under different illumination. (a) General light; (b) Strong sunshine; (c)Dusk ; (d) Night.

Table 3-3 Results of lane width estimation in the four situations

Curve (A) (B) (C) (D)

Mean (m) 3.422 3.421 3.423 3.424

Standard deviation (m) 0.1221 0.0874 0.1217 0.0885 Average error (m) 0.022 0.021 0.023 0.024 2) Lane width refinement

In lane detection, the initial settings are based on general width of lanes, i. e. 3m-5m, and the actual lane widths will later be adaptively refined based on the detected positions of the left and right lane markings in the image. Besides, to promote the robustness of lane width refinement, a Kalman filter is also adopted to stabilize and refine the process of lane width estimation.

Figure 3-15 shows the estimated lane widths with different preset widths and with/without Kalman filters in the sequential frames, where curve ‘(A) original 3m’and ‘(C) original 5m’

respectively represent the estimated lane widths with initial lane widths in 3m and 5m. The initial lane width of curve (A) and (C) were respectively set to be 3m and 5m. The curve ‘(B) Kalman 3m’and ‘(D) Kalman 5m’ respectively denote the estimated lane widths of curve (A) and (C) refined by the Kalman filter. By observing those results, the estimated lane widths with different preset lane widths will finally be refined to be closer to the actual ones. The application of the Kalman filter ensures stable and robust estimate results of the lane widths in the world coordinates. Table 3-3 displays the mean, standard deviation and average errors of the estimated lane widths in curve (A), (B), (C), and (D), and the actual lane width is about 3.4m. As can be seen, the estimated results in sequential frames are all quite close to the actual lane width and all of the average errors are under 0.024m. Furthermore, when the initially set lane width changes within the range from 3-5m, the obtained estimation results are still similar and close to the actual lane widths. The results show that our approach of lane width refining is robust and accurate.

3) Results of adaptation to illumination conditions

Dg1, Dg2, and Dg12 are determined by a statistical search algorithm based on the following two principles. (a) All gray level of lane markings is higher than those of the ground. (b) The variations of the gray levels of the ground and lane markings are within a reasonably fixed range. To demonstrate that our approaches are robust and adaptive to changes of illumination, variations of the gray levels of lane markings and grounds under four different illuminations are analyzed. The results are shown as Fig. 3-16, where lanes and lane markings display different gray level and contrast under different illumination. As can be seen from this fact, the principles (a) and (b) are appropriately followed under different illumination conditions, and the proposed statistical search algorithm can correctly and adaptively determine Dg1, Dg2, and Dg12 under various illumination conditions. Table 3-4 displays Dg1, Dg2, and Dg12 obtained from the four sample road scenes under different illumination conditions in Fig. 3-16, where Dg1, Dg2, and Dg12 are adaptively adjusted with various illuminations. As shown in Figs.

3-17~3-21, the adaptively determined thresholds can provide satisfactory lane detection results under different illumination conditions.

Table 3-4 The obtained parameters under different illumination conditions Illumination conditions Dg1 Dg2 Dg12

(a)General light 35 60 55

(b)Strong sunshine 30 10 85

(c)Dusk 25 20 50

(d)Night 70 110 75

Figure 3-17 shows the conditions of curves and a slope. In these figures, the roads with sharp curves and slopes still can be described by B-spline with four segments. Figure 3-18 is the situations with occlusion of obstacles. (a) The near front vehicle occluded lane markings of both sides. (b) The vehicle occluded the right lane marking. (c) The vehicle moved back to the road center. (d) The vehicle occluded the left lane marking. (e) The vehicles approached lane markings. (f) Another vehicle occluded the right lane marking. The figures prove the

problem of occlusion can be solved by the proposed approaches. The information of the side which is not occluded can be used to substitute the occluded one. When both sides of the lane markings are occluded, then only the parts that are not occluded can be shown.

(a)

(b)

(c)

(d)

Fig. 3-17. (a) Curves; (b) A slope; (c)(d) A cloverleaf interchange.

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 3-18. (a)(b)(c)(d)(e)(f) Situations of occlusion with different obstacles.

Figure 3-19 displays the detection results at night in situations including roads with or without road lamps and textures on the road surface, roads with curves and occlusion.

(a)

(b)

(c)

(d)

Fig. 3-19. Results of the nighttime road scene. (a)(b) With road lamps; (c)(d) Without road lamps.

Figure 3-17~3-19 present that FSM can extract BFT in various situations regardless of the influences of patterns on the road surface and illumination, and B-spline with four sections is able to display a variety of road conditions.

(a)

(b)

(c)

(d)

Fig. 3-20. The detection results under strong sunlight. (a)(b) No occlusion of vehicles; (c)(d) with the occlusion of a vehicle.

Figure 3-20 is the detection result under strong sunlight. The proposed approach can correctly detect the lane markings without being influenced by the strong sunlight.

(a)

(b)

Fig. 3-21. The detection result of a motorcycle inside and outside the lane. (a) Inside; (b) Outside.

Figure 3-21(a) and (b) respectively show a clear discrimination of a motorcycle inside and outside the lane. Obstacles inside the lane will affect driving safety. However, most contemporary lane detection approaches may not be able to discriminate whether an obstacle is inside or outside the lane when obstacles appear near the lane so they cannot correctly detect lane markings. In contrast, the proposed approach can resolve the problem of obstacle occlusion to reconstruct correct lane markings.

3.4.2 Comparative Performance Evaluation

In this subsection, comparative experiments on Jung and Kelber’s method [46] and the proposed approach is conducted to evaluate their performances on lane detection under different conditions. The following is a comparison of acquiring BFT by FSM and other approaches:

Figures 3-22~3-26 are respectively the comparative results of Jung and Kelber’s [46] and our proposed approach under different situations, where (a) is Jung and Kelber’s approach and (b) is our proposed approach. Figure 3-22 is the condition that the lane markings are occluded with shadows, signs of braking and other vehicles. Jung and Kelber adopted Sobel edge features of lane boundaries, which left large gradient points in the thresholded edge image, as shown in Fig. 3-22(a), the surrounding vehicle may cause false detection in the edge feature extraction process and result in detective errors. As shown in Fig. 3-22(b), the proposed approach successfully extracts features of lane markings with the BFT detector. The end part of the reconstructed lane boundary is the position of the last BFT, and the missing part at the end of the left-side lane marking is reconstructed with the information of the lane width and of the right-side lane marking.

(a)

(b)

Fig. 3-22. Results of the road scene that the lane markings are occluded with shadows, signs of braking. (a) Results of Jung and Kelber’s [46]; (b) Results of the proposed approach.

(a)

(b)

Fig. 3-23. Results of road scenes with a curve lane and occlusion. (a)Results of Jung and Kelber’s [46]; (b) Results of the proposed approach.

Figure 3-23 shows the results of the road scene consisted of curve lane and occlusion.

Figure 3-23(a) shows that the lane markings obtained by Jung and Kelber’s method have errors occurring on curves of roads when edge features of vehicles are mis-detected as the lane markings. Figure 3-23(b) demonstrates that our BFT approach can compensate the influences of appearing vehicles.

(a)

(b)

Fig. 3-24. Results of the road scene under strong sunlight. (a)Results of Jung and Kelber [46];

(b) Results of the proposed approach.

Figure 3-24 displays the detection results under strong sunlight. In Fig. 3-24 (a), edge features of vehicles associated with significant gradient features under strong sunlight cause possibly wrong determination of lane features. As shown in Fig. 3-24(b), with the BFT method, the proposed approach will not capture positions without lane markings to avoid wrong judgments in the far end of the lane. Therefore, lane boundaries can be reconstructed successfully.

Figure 3-25 displays the detection results at night. In Fig. 3-25(a), larger gradient arouses detection errors because of the opposite vehicle light and the light reflection of the preceding vehicle. The proposed approach can detect lane markings efficiently and correctly as shown in Fig. 3-25(b), because it takes projective sizes and sequences of lane markings into consideration in capturing BFT.

Figure 3-26 shows the detection results of the road scene with an S-shape lane. In Fig.

3-26(a), the S-shaped lane cannot be completely reconstructed when Jung and Kelber applied a linear-parabolic model to reconstruct lane boundaries. Figure 3-26(b) demonstrates that the proposed approach can successfully reconstruct the S-shaped lane boundary.

As can be seen from the above comparative results, the proposed approach can obtain satisfactory detection results under different situations, such as different illumination conditions, curve roads, and occlusions. This is because lane markings are extracted by the proposed BFT detector, and extraction errors can be effectively reduced by the proposed dynamic calibration method, ROI determination strategy and fuzzy rule-based scheme, and road boundaries are effectively reconstructed by the B-spline technique. Besides, when both sides of lane markings do not exist, or are occluded at the farther parts of the road, the range of the reconstructed lane is determined by the actual visible position of the lane, so the information obtained from previous frames will not be misused to reconstruct false lanes and driving safety can be improved.

(a)

(b)

Fig. 3-25. Results of the nighttime road scene. (a). Results of Jung and Kelber’s [46]; (b) Results of the proposed approach.

(a)

(b)

(c)

(d)

Fig. 3-26. Results of the road scene with an S-shaped lane. (a)(b) Results of Jung and Kelber’s [46]; (c)(d) Results of the proposed approach.