GPS

GPS is a common device in the smart phone now. It can provide the coordinate of longitude and time information. We can get the speed and time information through Microsoft standard API and use it to do the automatic 2D distance measure calibration.

5.2 Auto Calibration

In this thesis, we proved an algorithm to do the distance measurement. This algorithm has an important supposition that is front vehicle is const speed cruise. For this reason, we cannot calibration it immediately, we need to static the parameters set which derived from the algorithm in a while. Finally, we chosen the maximum statistic one to be the calibration result.

The accuracy of GPS is not very exact. We do the calibration at once in one second interval.

The parameters got in an image are time, speed and vehicle coordinate in the image. Figure 5.1 is the sketch map of the got parameters.

Figure 5.1: The sketch map of calibration parameter

By the anti-perspective projection, people can map points on the 2D plane to 3D world system. But in our system, users do not need to input camera parameter and setup coordination.

It is hard to do the mapping directly by this theory. In Figure 5.2, we can see that the x-axis project to y-axis on 2D plane is approaching to the parabolic curve. Thus, we propose to use parabolic curve (Y − b) = aX to get the mapping of 2D plane in coordinate with 3D world coordinate system for distance measuring.

Figure 5.2: Perspective projection of ground plane and the relationship of parabolic curve.

Following is our derivation about how to get the parabolic curve approach from three images with 1 second interval by themselves.

Image t1 front vehicle position

Step1. We can get the acceleration “A” by the equation (5.1).

A = (5.1) Step2. The shift of our car “S” is got by equation (5.2).

S = V1 + A

S = V2 + A (5.2) Step3. Then solve the following equation (5.3) to get the parameter “a” and “b” in the

parabolic curve equation (Y − b) = aX .

( ) ( )

(5.3)

=>

a =^{( ) ( )}

a =^{( ) ( )} (5.4)

=>^{( ) ( )} =^{( ) ( )} (5.5)

=> b = (5.6)

In the calibration stage, we needed to do the parabolic curve approach for a while.

Because we expected that the front vehicle was with a constant speed and we had the acceleration. If not fitting in these conditions, the result will be wrong. Therefore, we need to do the approach to static the parameter “a” and “b” for finding the maximum candidate to be our parabolic curve approach result.

Chapter 6

Distance Measurement

After dealing with the parabolic approach curve parameter “a” in the front chapter, we now can get front vehicle distance easily. The decision stage is just to find the front vehicle in the sequencial input images and use the following equation (6.1) to get the real world distance.

X = ^{( )} (6.1) Figure 6.1 show that we can measure the distance of front car is 120 meter.

Figure 6.1: The example of front car distance measuring result.

Finally, using this distance to make a decision and do some warning alarms to driver when the car is too close to the front vehicle. In our implement, we can make a decision in 100 milliseconds. This should good enough to help driver to avoid some overtaking collision danger.

Chapter 7

Experiment

The aim of this thesis is to build up a Driver Assistance System which is suitable for mobile devices. Currently there are four smart phone systems on the market Windows Mobile, RIM, iPhone and Android. Though iPhone has a complete develop API supporting and well performance but it is belong to high-end smart phone. RIM is just popular at Europe and the developed tools are not very complete. Android was not release when we created this project.

So we chose Windows Mobile system to be our target developing system.

7.1 Hardware Environment Description

Even though Microsoft has define a camera module access path from Direct Show after windows mobile 5.0, but there are still not many window mobile system smart phones supporting this feature. After having done the many tests, we just found that the HTC has supporting Direct Show completely. Therefore, we chose HTC Touch Diamond P3702 to be our developing environment. Figure 7.1 is its appearance.

Figure 7.1: HTC Touch Diamond P3702

Table 7.1 shows the detailed description of the system, it consists of the processor, memory, camera and GPS.

Table 7.1: Hardware system information.

Camera 320 Mega Pixels with auto focus

GPS GPS and A-GPS

7.2 Software Environment Description

Microsoft has a complete windows mobile developer center website.

http://msdn.microsoft.com/en-us/windowsmobile/default.aspx Our software system is listed at following table 7.2.

Table 7.2: Software system information.

Component Information

OS Windows Vista SP1

IDE Visual Studio 2008 Professional with SP1

SDK Windows Mobile 6 SDKs

Synchronization Windows Mobile Device Center .

7.3 Get Image by Direct Show

Getting image from camera is very hard from the previous to the present. Generally, the smart phone developer needed to get the camera SDK from camera vender for camera application development. But it is not release to end users. So we need to get image by Direct Show. Direct Show is a convent technique, users only need to assemble the components and finish its work. In Windows Mobile 6 SDKs, it supported “Camera Driver ” and “Video Capture” filters for getting image from camera. But, unfortunately it just has “Video Render” filter for display. It lacked “Video Grab” filter for grabbing video from the pipe. Therefore, we need to create a “Grabber” filter by myself and insert it to the filters link. Figure 7.2 shows the framework of video flow.

Figure 7.2: The flow chart of our Direct Show filters link.

7.4 Experiment Result

Figure 7.3 shows the result of the online front vehicle detection. The figure 7.3 (a), (b), (c) and (d) are sequencial images. The green horizontal lines in the camera image are the tracking of the front vehicle. The red oblique lines in the camera image are the lane detection markers.

They provide the information of process stage, frame rate, speed, front vehicle distance and got satellite number. We can confirm that we processed a frame in 100 milliseconds by the frame rate. The front car in the sequence images can show that our distance measurement algorithm is match to the front car revealing in the images.

Camera

Camera Driver (DMO)

Video Capture (CLSID_Vide oCapture)

Grabber

Video Render

LCD

Process the image

(a) (b)

(c) (d) Figure 7.3: Front vehicle tracking and distance measure.

(a) 120 meters away from the vehicle in front.

(b) 132 meters away from the vehicle in front.

(c) 117 meters away from the vehicle in front.

(d) 106 meters away from the vehicle in front.

Chapter 8

Conclusions and Future Works

This study provided a driver assistance system on the smart phone. The experiment demonstrated that the algorithm was workable and had enough accuracy. But it still has many defects such as it could just be used in daytime and high way. The GPS latency and accuracy will cause the system collected with wrong results. The bad weather and lane marker will lead to fail decision. The non vehicle shadows lead vehicle to judge in the mistake. But, by the limitation of the computing power, we could just reach to this performance. If the computing power is good enough, we can design a better shadow pick out algorithm for fake shadows. If the memory is enough, we can keep the pre-frames in the memory to overcome GPS latency. The decision system and training system are required to be redesigned for nighttime as well.

Reference

[1] S. K. Kemue, “LANELOK: Detection of lane boundaries and vehicle tracking using image-processing techniques – Parts I and II” SPIE Mobile Robots IV, 1989.

[2] K.C. Kluge, YARF: An Open-Ended Framework for Robot Road Following, PH.D Thesis, Carnegie Mellon University, 1993.

[3] K.C. Kluge, “Extracting road curvature and orientation from image edge points without perceptual grouping into features,” Proceedings of the Intelligent Vehicle ’94 Symposium, pp.109-114, 1994.

[4] D. Pomerleau and T. Jochem, “Rapidly Adapting Machine Vision for Automated Vehicle Steering,” IEEE Expert, 11, (2), pp.302-7, April 1996.

[5] K. C. Kluge, “Performance evaluation of vision-based lane sensing: some preliminary tools, metrics and result,” IEEE Conference on Intelligent Transportation System, 1997.

[6] K. Y. Chiu and S. F. Lint, “Lane Detection using Color-Based segmentation”, Department of Electrical and Control Engineering, National Chiao-Tung University, Taiwan, ROC.

[7] S. Terakubo, et. al., “Development of an AHS safe driving system,” SEI Technical Review, no. 45, pp.71-77, 1998.

[8] M. Y. Chern, B. Y. Shyr, "Locating nearby vehicles on highway at daytime based on the front vision of a moving car," Proceedings of Robotics and Automation, vol. 2, 2003, pp.

2085 - 2090.