Chapter 1 Introduction
1.2 Motivations
The most common current practice in violation enforcement involves determining and sending out written notices for each violation. Since the accuracy of ALPR is not always reliable, human review and correction will be needed to improve the accuracy of the license plate reading. However, recognizing a large number of photographs will cause a large number of human reviews and corrections will increase the cost of ETC system. In SLFF, it is easy to match the vehicle’s transaction record to its image record correctly because both of these record must take place at the same lane; however, in MLFF, it is relatively complicated to match the vehicle’s transaction record to its image record correctly because of the vehicle may change the lane and it may occur that transaction record can’t match with image record.
In this thesis, we focus on improving the efficiency of VES. In order to solve the problem of matching the correct transaction data to correct license plate image data, we propose a bipartite graph matching solution which not only can reduce the human loading but also can accurately identify the vehicles that drive through a highway toll gate area without paying for toll. To match the vehicle’s transaction data to its license plate image data correctly, ALPR technology is one of the important factors. As mentioned above, the precision of the ALPR is not reliable. To reduce the count of the ALPR, we propose the bipartite-graph-based algorithm to cope with the bipartite graph. Simulation result shows that our method doesn’t need to recognize each picture, so that it can increase the accuracy to verify the illegal vehicle which doesn’t pay the toll.
6
The rest of this thesis is organized as follows. In chapter 2, we reviewed the related works. Then, we introduce that how to construct the bipartite graph and utilize the divide-and-conquer algorithm to decrease the count of ALPR. Chapter 4 shows our simulation results and analysis. Finally, this thesis is concluded in Chapter 5.
7
Chapter 2
Related Works
ETC needs serious accuracy and feasible. Several studies have been proposed to design the architecture for ETC [4]. Some researches were proposed for AVI components.
In [5], the authors designed in-pavement antennas with carrier frequency 915MHZ. The tag is on the lower edge of the front license plate, and the in-pavement antennas are buried under the road. The transmission range is one meter wide and 2 meter high. It support variable bits packet for several operations. However, it is not sufficient for congested traffic. In [6] the authors proposed MLFF architecture for ETC. The gantry is 6.2m height cross the width of three lanes and the transceivers are with carrier frequency 5.8GHz. In [7] the authors proposed a novel architecture by employing millimeter-wave range in MLFF. Each lane is equipped with antennas and the frequency of each is different. The proposed scheme utilized high resolution in lateral directions to track the vehicle’s direction. This is for separating the packets into segments so that can communicate with RSU consecutively. The communication range may overlap on the intersection of adjacent lane.
Some researches were proposed for AVC components. In [8], the authors proposed a wire device that can get the electronic signal when vehicle passing. The device can classify type of vehicles by the variation signal information. ETC System can utilize the information to toll different amount of money.
8
Some researches were proposed for Enforcement System. In [9] proposed VPS scheme. As shown in Figure 3, the toll zone is a rectangle area on each direction and covers the range the vehicles may pass. The coordinate of virtual toll is stored on OBU, so vehicles can know whether they drive in the area by GPS. The area has two parts: Notification area and toll area. When vehicles pass through the notification area, OBU is ready for paying tolls. The vehicle would send transaction request message to back server in RSU. The vehicle’s plate number is sent to RSU for matching the license image. However, ALPR is not reliable and it has to recognize every image.
Notification
area Toll area
Figure 3 Virtual toll zone configuration in VPS system.
In [10], the authors proposed the enforcement system, the system using the transaction to find the mapping image data. If not found, the system would keep find according the order of lane. If still not found, entering the human checking review. In [11], the authors proposed the method to increase the success rate to matching the image data and transaction data. As shown in Figure 4, there are two report points. At these points, the vehicle would report his current coordinate to RSU. The two coordinates can compute the position where the vehicles are taken pictures. Servers can just only check the image data that located at the possible lane. However, the
9
method is still to recognize all the image data so that to map the transaction data. The follow chapter would introduce our algorithm that is efficient and doesn’t need recognize all the image data.
(x1,y1,t1,v1)
(x2,y2,t1,v2) Lane 1
Lane 2
Lane 3
Lane 4
Figure 4 Using two report points to evaluate the lane position
10
Chapter 3
Bipartite-Graph-Based Algorithm
In this chapter, we introduce our bipartite-graph-based algorithm. In section 3.1, the system model is described. In section 3.2, we transfer our problem into bipartite graph and introduce the characteristic of the graph. In section 3.3, utilizing divide-and-conquer algorithm to cope with bipartite graph decreasing the count of ALPR. In section 3.4, is the whole view of our algorithm.
3.1 System Model
In the system model, we suppose the scenario of the road is a highway with 4-lanes considering single direction. In this thesis, we assume of the vehicles on the road equip with positioning devices such as GPSs to acquire their own position. There is a RSU to provide the AVI service and several cameras on each lane in the toll collection plaza.
We supposed that the communication between RSU and OBU is 802.11p/1609.
Utilizing WAVE/DSRC has more extensive transmission range than DSRC-based ETC system.
Different from the traditional architecture, the vehicles can communicate within the RSU’s transmission range in our model as shown in Figure 5 ( the area of the
11
dotted ellipse ), it means that the positions of tolls which occur in the range and are not restricted before the RSU.
3.2 Bipartite Graph
A bipartite graph is an undirected graph G = (V, E) which vertices can be partitioned into two disjoint sets V1 and V2 such that (u, v) ∈ E, u ∈ V1 and v ∈ V2.
The primary step is to create a bipartite graph representation of the relation between license plate image data and transaction data. The flow for creating a bipartite graph is described as follow: As shown in Figure 5, consider a vehicle equipped with an OBU is driving on a highway. When the vehicle enters the toll gate area defined by a RSU, the OBU can use wireless communication to pay the toll to the RSU and then the RSU sends the transaction data to the Enforcement System. The transaction data includes the time of transaction and license plate ID information. When the vehicle passes through the Enforcement system, the Enforcement System
Figure 5. An image data and the possible matching targets
12
will take a picture of the license plate in order to get the license plate image data. At the same time, the Enforcement System gets the speed (V2) of the vehicle and uses the time of taking the picture (T1) to infer the time of entering the toll gate area (T2) and
Let the speed of vehicle which enter the communication (V1) is equal to zero, and the transmission area of the RSU is S, then we can infer the threshold time of passing the transmission area to ensure that all the image data could map to whole possible transaction data. All the transaction data from T3 to T2, said the possible matching targets, will be connected to the image data. As the shown example in Figure 5, there are five transaction data from T3 to T2. So the image data has five possible matching targets. As shown in Figure 6, for each image data, we can connect it with the corresponding transaction data and then a maximum connected bipartite graph G (PT, E) is created, where the set of vertices P represents image data, the set of vertices T represents transaction data and the set of edges E represents links between P and T.
Figure 6. A bipartite graph between image data and transaction data.
13
3.2.1 Graph property
Let G(V, E) be a graph and vV(G) be a vertex. Given P ={𝑃1, 𝑃2, ……, 𝑃𝑚} set of image data, T={𝑡1, 𝑡2, ……, 𝑡𝑛} set of transaction data and 𝐸={(𝑃𝑖, 𝑇𝑗)│there exists some possible transactions are the matching targets of license plates according velocity acquired during taking pictures.}
We use the notation N(v) and E(v) to denote the set of vertices connected to v and the set of edges incident with v, respectively. The cardinality |E(v)| is called the degree of v, denoted by deg(v). Consider a maximum connected bipartite graph G(PT, E) representing the relation between license plate image data and transaction data. Then G(PT, E) has the following properties:
Property 1. The number of vertices of the image data set P is greater or equal to the number of vertices of the transaction data set T.
1.1 If |P| = |T|, all the vertices in P are legal vehicles. It means that all these vehicles drive through the toll gate area with paying for toll.
1.2 If |P| > |T|, there exists at least one illegal vehicle in P without paying for toll.
Property 2. For a vertex v in P, if v is a legal vehicle, the transaction data of v must be in N(v). It means that the transaction data of v is in the set of possible matching targets of v. As shown in Figure 7.
Figure 7. An example of the transaction data of v must be in N(v).
Property 3. For a vertex v in P, if deg(v) = 0, v must be an illegal vehicle.
14
Property 4. For a vertex u in T, the image data of u must be in N(u). As shown in Figure 8.
Figure 8. An example of the iamge data of u must be in N(u)
Property 5. For a vertex u in T, if deg(u) = 1 and vertex v is the only neighbor of u, v is a legal vehicle and u is the transaction data of v.
Figure 9. (a) Initial bipartite graph (b) The graph is processed by Property 5.
Consider an example of a bipartite graph as shown in Figure 9(a), the transaction data with the license plate ID is P01802, B00010, and A21787, each of them has the only one in-degree, so we can say that plate1 is mapping to P01820, plate2 is mapping to B00010, and plate3 is mapping to A21787, then, we can remove irrelevant edges which are connected to the transaction data. Figure 9(b) shows the result after processing by Property 5.
3.2.2 Simplify Bipartite Graph
In the congested environment, the speed of vehicles is relative slow. It means that there are numerous vehicles stay in the transaction area at the same time. Thus,
(a) (b)
15
each of the image data could map to as much as possible transaction data. We assume all of the vehicles that equip OBUs would report their own position to server during paying tolls. As shown in Figure 10, there are two types of the charging vehicles in the transaction area. One is charging before the camera, and the other is after the camera.
Figure 10. Two types of the charging vehicle
The image data time of vehicles which are charging before Enforcement System must occur after the time of transaction data; in the same way, the image data time of vehicles which are charging after Enforcement System must occur before the time of transaction data. Using the above characteristic could eliminate unnecessary image data’s out-degree to transaction data. Consider an example as shown in Figure 11, The red circles are the vehicles which paid tolls after Enforcement System and the blue circle is the vehicle which paid tolls before the Enforcement System. The time of the blue circle is 00:00:05, so the time of possible image data must after 00:00:05, so the blue circle has only one in-degree that connected to the plate5. The time of the red circles are 00:00:03 and 00:00:01 each, so the time of possible image data are plate1, plate2 and plate3.
In addition to the time of charging position, we apply Figure 4 to predict the lane
16
ID where the vehicles are taken images on, then we classify the image data and transaction data according to the lane ID, for example the number of lanes are four, we can divide the original bipartite graph into four subgraph, this would decrease the number of degree of each vertex in G. Reducing the number of degrees would improve our algorithm performance, the impact of the factor will investigate in Chapter 4 and compare to the original bipartite graph, partition graph by charging position and partition by lane ID.
(a) (b)
Figure 11. (a) Initial graph (b) After eliminate degree
3.3 Bipartite-Graph-Based Algorithm
We design an algorithm to discover the illegal vehicles which are not equipped OBUs efficiently by utilizing the graph property above mentioned. As shown in Figure 12, the main spirit of the algorithm here is the use of Property 1. When the number of vertices of the image data set is equal to the number of vertices of the transaction data set, it can determine all the image data are legal vehicles quickly and reduce the number of image recognitions. On the other hand, if the number of image data is greater to the number of the transaction data, there exists at least one illegal vehicle in image data set. In this case, the image recognition processing can make that the two sets have the same number of vertices or disconnect the bipartite graph into disjoint components. For each component, it is recursive and used as the input of the
17
algorithm.
Figure 12. The pseudo code of PT algorithm.
The details are described as follows :
line 1 and 2: Explain the input and output.
line 3: The input is a maximum connected bipartite graph G(PT, E)
line 4 ~ line 6: Apply Property 1.1 to check whether the number of vertices of 1 Input: a maximum connected bipartite graph G(PT, E).
2 Output: two sets of illegal vehicles and legal vehicles.
3 Input (G(PT, E)) {
10 run Check_Degree_One_Vertices_In_T;
11 If G is disconnected and has G1, G2, …, Gn components
18
image data set is equal to the number of vertices of transaction data set or not. If agree, all image data are legal and moved to the set of legal vehicles.
line 7 ~ line 9: Apply Property 3 to check whether the number of vertices of image data set is equal to one and the number of vertices of transaction data set is equal to zero. If agree, the only image data is illegal and moved to the set of illegal vehicles.
line 10: Run the Check_Degree_One_Vertices_In_T function. By Property 5, for a vertex in transaction data set with degree one, the only neighbor of this vertex must be legal and can be moved to the set of legal vehicles.
line 11 ~ line 12: If G is disconnected and has some components after the steps above, the components can be recursive and used as the input of the algorithm.
line 13 ~ line 29: Use the image recognition processing repeatedly until the two sets of image data and transaction data have the same number of vertices or G is disconnected.
Figure 13 Demonstrates the detail flow chart of the Bipartite-Base Algorithm.
19
（1)The number of photo = The number of transaction (2) The number of transaction = 0
Graph is still
Figure 13. Flow Chart of Bipartite-Based Algorithm
20
We take two examples to explain the whole Bipartite-Base Algorithm. Consider the set of image data have nine vertexes, and the set of transaction have eight vertexes, so it means that there is an illegal vehicle. To find the illegal vehicle, taking the middle vertex plate5 of the Photos set to execute plate recognition first, the car ID of plate5 is A21787 as shown in Figure 14(a). The degrees of A21787 exist mapping ID, so it’s illegal. Second step is removing the vertex from Photos set, and eliminating the degrees which are connected to transaction data A 21787. The bipartite graph is still connective and the number vertexes of Photo set are still more than the number vertexes of Transaction set, so taking the middle vertex plate4 of the Photos set to execute plate recognition again, and the car ID of plate4 is B01610 as shown in Figure 14(b). The degrees of B01610 don’t exist mapping ID, so it’s an illegal vehicle and removing the vertex from Photos set. Finally, the number vertexes of each set are same, so the remain vehicles are all legal.
Figure 14. (a)Plate5 is recognized as legal (b)Plate6 is recognized as illegal
Consider another example as shown in Figure 15. We assume that B01610 is legal now, then the vertex B01610 of Photos set is removed, and all the edges which are connected to the vertex B01610 of Transaction set are removed too. The bipartite
(a) (b)
21
graph is disconnected as shown in Figure 15 (b). There are two subgraph, one of has four vertexes of each set, so remain four vehicles are all legal. On the other hand, another subgraph has three vertexes in Photos set and two vertexes in Transaction set, it means that there is one illegal vehicle in the subgraph. The components can be recursive and used as the input of the algorithm to find remain illegal vehicle.
Figure 15. (a) B01610 is legal (b) Remain two bipartite subgraph
(a) (b)
22
Chapter 4
Simulation Result and Analysis
4.1 Simulation Environment
In this chapter, we show the simulation result of our algorithm and its performance.In [12], the mobility generator has including Random Waypoint model, Reference Point Group Mobility model, Reference Point Group Mobility model and Man-attan mobility model. It is used in our simulations to generate mobility scenario for the Freeway Model. As mentioned in Chapter 3, the simulation scenario is in an 4km highway with 4 lanes considering single direction. We conduct the simulation using ns-2 simulator [14]. We evaluate our alogrithm on three scenarios for congested, normal, and sparse traffic. As mentioned in 3.2.2, we compare the ALPR count of orginal bipartite graph and simplified graph.
The tolerant of the traffic flow in Taiwan ETC is 2,210 vehicles/hour in the rush hour [13], as shown in Figure 16, the most vehicles appear at 10am to 21pm, and the total traffic flow is almost 8000 vehicles an hour for all lanes. Electronic toll collection have been implemented in Taiwan for five years, the proportion of illegal vehicles which are not equipped OBUs to total traffic flow as show in Figure 16. The average error rate is closed to 0.06% recent years. In our simulation, we utilized the error rate from 0.06% to 3.0% in the three scenarios.
23
Figure 16 Traffic flow at Taishan Toll Station a day.
http://211.79.135.72/volume/drawday.htm
Figure 16 Traffic flow at Taishan Toll Station a day.The proportion of illegal vehicles
We have three different scenarios: congested traffic, normal traffic, and sparse traffic. The main different parameters between these scenarios are the velocity, number of vehicles and acceleration. Table 1 showed our simulation parameters details.
24
The Length of Highway 4000m
Number of lanes 4 lanes
Number of Vehicles [4410, 3286, 2519]
Velocity Vmax:[108, 80, 54]Km/h
Vmin:[90, 72, 36] Km/h Acceleration Speed [0.4, 0.8, 1.2]m/s2
Simulation times 1800s
Transmission Range 100m
Location of the camera 2000m
Location of the RSU 2000m
Location of the RSU 2000m