Chapter 3 Accurate Lane Detection
3.6 Lane Voting
In this thesis, we proposed to use the lane voting algorithm to get an accurate lane model equation. The benefit of lane voting algorithm is that we do not need to get an absolutely clear lane marker image but can get accurate lane model equation easily. The first stage is only to get lane marker image and following with the follow chart of getting pre-process lane image for the lane voting.
Figure 3.4: The flow chart of pre-process lane image for lane voting.
In Figure 3.4, the stages of “Image Normalization”, “Lane Detection in HSV Color Domain” and “Apply Lane Filter” are related to previous chapters.
After getting the lane mark images, we can start to do the lane voting process. From chapter 3.4, we have got two equations. One is the left lane model and another is right. The idea is to use these equations to scan full lane mark image and take the maximum matched equation to be the lane model equation. (Figure 3.5)
Get Color Image
Image Normalization
Lane Detection in HSV Color Domain
Apply Lane Filter
.
Figure 3.5: (a) Matching left lane. (b) Matching right lane.
. (a)
(b)
Figure 3.5: (a) Matching left lane. (b) Matching right lane.
.
.
…
.
… … …
Figure 3.5: (a) Matching left lane. (b) Matching right lane.
After getting the maximum matching, we matched lane model is just the closest equation
we fix the parameter “b” in the equation (3.3) and do some addition parameter “a” to get potential maximum match.
the maximum matching, we need to do some angle fine tun matched lane model is just the closest equation to the real lane marker. In our fine tune
in the equation (3.3) and do some additions and subtraction to get potential maximum match. In Figure 3.6 is our matching result.
Figure 3.6: Lane voting result.
need to do some angle fine tunings. The real lane marker. In our fine tune process,
and subtractions on igure 3.6 is our matching result.
The vehicle detection is very important in this system.
in the image demands a complexity algorithm to do it. In this algorithm to detect a vehicle in the
4.1 Detect Vehicle by the Shadow
In the chapter 3.1, we separated
stage. In this chapter, we need to do more calculation to extract the vehicle at the front. We do not care the neighborhood vehicle
up of the lane models. The blue area in the following
4.2 C-means
We used shadow to conjecture shape feature so it is hard to use
the shape. Here, we use C-means to decide the position of algorithm.
V(x, y) = ∑ xi , ∑
N: The markers of vehicle in the searching area.
In our system, we just need to get the y position. So, we can simplify the equation (4.1) as following.
V(y) = ∑ yi
Figure 4.2 is our search result.
Chapter 4
Vehicle Detection
ehicle detection is very important in this system. A robust vehicle detection process a complexity algorithm to do it. In this thesis, we propose
in the front, but it bases on the searching assistance
separated the land and vehicle images at image color transform stage. In this chapter, we need to do more calculation to extract the vehicle at the front. We do
t care the neighborhood vehicles but just the front one. So, the search area is just in the up of the lane models. The blue area in the following Figure 4.1 is the searched area.
Figure 4.1 Vehicle searched area.
conjecture the front vehicle, however the shadow t is hard to use feature definition to make decision about the
means to decide the position of the front vehicle.
yi N: The markers of vehicle in the searching area.
In our system, we just need to get the y position. So, we can simplify the equation (4.1) as
A robust vehicle detection process , we proposed an easy assistance at pre-chapter.
at image color transform stage. In this chapter, we need to do more calculation to extract the vehicle at the front. We do but just the front one. So, the search area is just in the fence
igure 4.1 is the searched area.
the shadow do not have any to make decision about the authenticity of front vehicle. Following is our
(4.1)
In our system, we just need to get the y position. So, we can simplify the equation (4.1) as
(4.2)
(a) (b) Figure 4.2: (a) Real image. (b) Vehicle markers image.
Chapter 5
Auto Calibration
Using computer vision to detect distance from 2D image in previous papers is very complex for users. Generally, it is hard to detect image from 2D image directly. People need to provide camera parameter and the coordinates of camera setup. We provide a user friendly distance measure method assists with GPS.
5.1 GPS
GPS is a common device in the smart phone now. It can provide the coordinate of longitude and time information. We can get the speed and time information through Microsoft standard API and use it to do the automatic 2D distance measure calibration.
5.2 Auto Calibration
In this thesis, we proved an algorithm to do the distance measurement. This algorithm has an important supposition that is front vehicle is const speed cruise. For this reason, we cannot calibration it immediately, we need to static the parameters set which derived from the algorithm in a while. Finally, we chosen the maximum statistic one to be the calibration result.
The accuracy of GPS is not very exact. We do the calibration at once in one second interval.
The parameters got in an image are time, speed and vehicle coordinate in the image. Figure 5.1 is the sketch map of the got parameters.
Figure 5.1: The sketch map of calibration parameter
By the anti-perspective projection, people can map points on the 2D plane to 3D world system. But in our system, users do not need to input camera parameter and setup coordination.
It is hard to do the mapping directly by this theory. In Figure 5.2, we can see that the x-axis project to y-axis on 2D plane is approaching to the parabolic curve. Thus, we propose to use parabolic curve (Y − b) = aX to get the mapping of 2D plane in coordinate with 3D world coordinate system for distance measuring.
Figure 5.2: Perspective projection of ground plane and the relationship of parabolic curve.
Following is our derivation about how to get the parabolic curve approach from three images with 1 second interval by themselves.
Image t1 front vehicle position
Step1. We can get the acceleration “A” by the equation (5.1).
A = (5.1) Step2. The shift of our car “S” is got by equation (5.2).
S = V1 + A
S = V2 + A (5.2) Step3. Then solve the following equation (5.3) to get the parameter “a” and “b” in the
parabolic curve equation (Y − b) = aX .
( ) ( )
(5.3)
=>
a =( ) ( )
a =( ) ( ) (5.4)
=>( ) ( ) =( ) ( ) (5.5)
=> b = (5.6)
In the calibration stage, we needed to do the parabolic curve approach for a while.
Because we expected that the front vehicle was with a constant speed and we had the acceleration. If not fitting in these conditions, the result will be wrong. Therefore, we need to do the approach to static the parameter “a” and “b” for finding the maximum candidate to be our parabolic curve approach result.
Chapter 6
Distance Measurement
After dealing with the parabolic approach curve parameter “a” in the front chapter, we now can get front vehicle distance easily. The decision stage is just to find the front vehicle in the sequencial input images and use the following equation (6.1) to get the real world distance.
X = ( ) (6.1) Figure 6.1 show that we can measure the distance of front car is 120 meter.
Figure 6.1: The example of front car distance measuring result.
Finally, using this distance to make a decision and do some warning alarms to driver when the car is too close to the front vehicle. In our implement, we can make a decision in 100 milliseconds. This should good enough to help driver to avoid some overtaking collision danger.
Chapter 7
Experiment
The aim of this thesis is to build up a Driver Assistance System which is suitable for mobile devices. Currently there are four smart phone systems on the market Windows Mobile, RIM, iPhone and Android. Though iPhone has a complete develop API supporting and well performance but it is belong to high-end smart phone. RIM is just popular at Europe and the developed tools are not very complete. Android was not release when we created this project.
So we chose Windows Mobile system to be our target developing system.
7.1 Hardware Environment Description
Even though Microsoft has define a camera module access path from Direct Show after windows mobile 5.0, but there are still not many window mobile system smart phones supporting this feature. After having done the many tests, we just found that the HTC has supporting Direct Show completely. Therefore, we chose HTC Touch Diamond P3702 to be our developing environment. Figure 7.1 is its appearance.
Figure 7.1: HTC Touch Diamond P3702
Table 7.1 shows the detailed description of the system, it consists of the processor, memory, camera and GPS.
Table 7.1: Hardware system information.
Camera 320 Mega Pixels with auto focus
GPS GPS and A-GPS
7.2 Software Environment Description
Microsoft has a complete windows mobile developer center website.
http://msdn.microsoft.com/en-us/windowsmobile/default.aspx Our software system is listed at following table 7.2.
Table 7.2: Software system information.
Component Information
OS Windows Vista SP1
IDE Visual Studio 2008 Professional with SP1
SDK Windows Mobile 6 SDKs
Synchronization Windows Mobile Device Center .
7.3 Get Image by Direct Show
Getting image from camera is very hard from the previous to the present. Generally, the smart phone developer needed to get the camera SDK from camera vender for camera application development. But it is not release to end users. So we need to get image by Direct Show. Direct Show is a convent technique, users only need to assemble the components and finish its work. In Windows Mobile 6 SDKs, it supported “Camera Driver ” and “Video Capture” filters for getting image from camera. But, unfortunately it just has “Video Render” filter for display. It lacked “Video Grab” filter for grabbing video from the pipe. Therefore, we need to create a “Grabber” filter by myself and insert it to the filters link. Figure 7.2 shows the framework of video flow.
Figure 7.2: The flow chart of our Direct Show filters link.
7.4 Experiment Result
Figure 7.3 shows the result of the online front vehicle detection. The figure 7.3 (a), (b), (c) and (d) are sequencial images. The green horizontal lines in the camera image are the tracking of the front vehicle. The red oblique lines in the camera image are the lane detection markers.
They provide the information of process stage, frame rate, speed, front vehicle distance and got satellite number. We can confirm that we processed a frame in 100 milliseconds by the frame rate. The front car in the sequence images can show that our distance measurement algorithm is match to the front car revealing in the images.
Camera
Camera Driver (DMO)
Video Capture (CLSID_Vide oCapture)
Grabber
Video Render
LCD
Process the image
(a) (b)
(c) (d) Figure 7.3: Front vehicle tracking and distance measure.
(a) 120 meters away from the vehicle in front.
(b) 132 meters away from the vehicle in front.
(c) 117 meters away from the vehicle in front.
(d) 106 meters away from the vehicle in front.
Chapter 8
Conclusions and Future Works
This study provided a driver assistance system on the smart phone. The experiment demonstrated that the algorithm was workable and had enough accuracy. But it still has many defects such as it could just be used in daytime and high way. The GPS latency and accuracy will cause the system collected with wrong results. The bad weather and lane marker will lead to fail decision. The non vehicle shadows lead vehicle to judge in the mistake. But, by the limitation of the computing power, we could just reach to this performance. If the computing power is good enough, we can design a better shadow pick out algorithm for fake shadows. If the memory is enough, we can keep the pre-frames in the memory to overcome GPS latency. The decision system and training system are required to be redesigned for nighttime as well.
Reference
[1] S. K. Kemue, “LANELOK: Detection of lane boundaries and vehicle tracking using image-processing techniques – Parts I and II” SPIE Mobile Robots IV, 1989.
[2] K.C. Kluge, YARF: An Open-Ended Framework for Robot Road Following, PH.D Thesis, Carnegie Mellon University, 1993.
[3] K.C. Kluge, “Extracting road curvature and orientation from image edge points without perceptual grouping into features,” Proceedings of the Intelligent Vehicle ’94 Symposium, pp.109-114, 1994.
[4] D. Pomerleau and T. Jochem, “Rapidly Adapting Machine Vision for Automated Vehicle Steering,” IEEE Expert, 11, (2), pp.302-7, April 1996.
[5] K. C. Kluge, “Performance evaluation of vision-based lane sensing: some preliminary tools, metrics and result,” IEEE Conference on Intelligent Transportation System, 1997.
[6] K. Y. Chiu and S. F. Lint, “Lane Detection using Color-Based segmentation”, Department of Electrical and Control Engineering, National Chiao-Tung University, Taiwan, ROC.
[7] S. Terakubo, et. al., “Development of an AHS safe driving system,” SEI Technical Review, no. 45, pp.71-77, 1998.
[8] M. Y. Chern, B. Y. Shyr, "Locating nearby vehicles on highway at daytime based on the front vision of a moving car," Proceedings of Robotics and Automation, vol. 2, 2003, pp.
2085 - 2090.