Chpter 4 Evaluation and Analysis
4.1 Simulation Environment and Parameters
The scenarios presented in this thesis simulate the traffic of an urban arterial road of 2.4 km in HsinChu city, Taiwan. In order to evaluate the efficiency of the system we proposed, we have developed a simulation system that uses the traffic model simulator VISSIM [15]. Table 4-1 shows an example of sequence data generated by VISSIM. The data record consists of a record of time, a probe car’s identifier, a probe car’s current speed, and an identifier of the road segment that the probe car drives on.
Using these data, we can obtain enough traffic information of the probe cars. The simulation system allowed us to obtain a detailed analysis of the performance of proposed methods.
Table 4-1 The sequence data generated by VISSIM.
Simulation time ID of probe car Speed (kph) Segment ID
302 278 46.4 1
302 20 45.8 3
303 278 45.5 1
303 20 45.3 3
304 278 44.6 1
304 20 44.7 3
Table 4-2 lists the parameters used for the simulation we have developed. In the simulation, we simulate a traffic situation which reflects the entire period of traffic congestion, i.e., the traffic flow changes from light to heavy to light again. In this simulation, traffic congestion can be divided into 3 intervals: (1) congestion emergence for 700 seconds, (2) congested traffic for 1300 seconds, and (3) congestion
29
disappearance for 1600 seconds.
Table 4-2 Parameters used in the simulation.
Total road length 2400 m
Number of lanes 3
Number of segments 4
Desired Speed 40 ~ 50 kph (uni. distributed) Composition of traffic 20% probe cars, 80% regular
Broadcast interval (T) 30 (s)
Evaluation time 3600 (s)
Traffic flow 1500 -> 2000 -> 800 (veh/hr)
4.2 The Maximum Speed
In the simulation, each probe car updates its speed every second. In order to reflect overall traffic conditions of a segment, these speed data are averaged with previous 30 instantaneous speeds. The reason is that simply using instantaneous speed data would only reflect partial traffic conditions, especially in the areas near road intersections. For example, a probe car would drive at high speed for a moment when the traffic light turns green. In this case, the maximum instantaneous speed does not reflect the general traffic condition of whole segment. However, average data set represents the traffic situation of a period of time, generating proper traffic information.
In the experiments of the traffic information system providing Vmax, we evaluate the performance of our system at first. We compare our system with conventional system using periodical policy. In the case of periodical policy, each probe car reports its Vcmax to the TIC periodically. The report cycle is set to be 30 seconds, which is the same as the TIC’s broadcast interval. In addition, each probe car reports when entering a new segment. To compare both systems, we use two metrics to measure
30
them during the evaluation of simulations. One is the communication cost, which is defined as the amount of messages probe cars report per period. The other is the error, which is calculated as the mean difference between the real Vmax and the computed Vmax; in other words, the error is defined as the mean absolute error (MAE). In this experiment, the broadcast latency of the TIC is set to be half of a second, and the parameters used in the report policy, α and β, are chosen from experiment. In order to obtain the minimum communication cost, these parameters are set to 1.1 and 1.4 respectively. Table 4-3 shows the comparison of conventional system and proposed system.
Fig. 4-1 plots the comparison of total amount of messages obtained by different values of α with varying values of β. This graph shows that our system generates the minimum amount of messages when α is 1.1. The reason is that the bigger α induces the longer length of timer TC, and the time difference between tend and trecv is smaller.
Therefore, the length of timer Tm-tmp is short. Several probe cars almost report Vcmax at the same time because their timer lengths are similar. We also observe that the amount of message is high when β is small, and the amount of message is high when β is big as well. If β is small, then probe cars would report frequently when accelerating. On the other hand, big β induces a long length of timer Tm-tmp. As a result, probe cars report Vcmax almost at the end of current period. Since broadcast has latency, these probe cars report Vcmax before receiving Vtmp, resulting in growing amount of messages.
Table 4-3 The comparison of convention and proposed system.
Segment Time Vmax
Convention Proposed
Vmax Report
Count Vmax Report
Count
31 significantly better than the conventional one. The communication cost of the conventional system was 24.875, while the proposed one only generated 2.3 messages per period. In terms of the error metric, the proposed system produced no error, i.e., it can generate the actual Vmax. The findings indicate that our system not only reduces communication requirements but also maintains the real-timeliness of generated traffic information.
32
Table 4-4 The results of performance.
Method Communication cost
(messages/period)
MAE
Convention 24.875 1.525
Proposed 2.3 0
In the next part of the experiments, we examine the accuracy of our system to show that we can use Vmax as a good metric for detection of traffic states. First of all, we have to determine traffic states by using average speed, which is denoted as Vavg. The reason is that traffic engineers conventionally utilize average speed to analyze the efficiency of transportation, and they commonly use the level of service (LOS) to measure and determine the traffic state. Based on the Highway Capacity Manual in Taiwan [16], we can quantitatively classify Vavg into three levels as follows:
(1) Red: Vavg < 16 (kph),
(2) Yellow: 16 ≤ Vavg < 25 (kph),
(3) Green: 25 ≤ Vavg,
where red level stands for traffic jam, yellow stands for heavy traffic, and green level represents light traffic. The second step is to determine traffic states by using Vmax. We use two adjusting thresholds to classify the data of Vmax generated by the TIC. One is Vyellow, and the other is Vgreen. These two thresholds are calculated according to the empirical data of average speed. Vyellow is computed as the minimum Vmax during the period of green level, and Vgreen is computed as the minimum Vmax during the period of yellow level. Table 4-5 shows an example of selection of Vyellow and Vgreen. According to the rule, Vyellow is chosen to be 36, and Vgreen is chosen to be 41. To get the best
33
overall accuracy, a slight adjustment may be needed. Table 4-6 shows an example of adjustment. The accuracy improves when Vgreen is adjusted to be 42. It should be noted that both Vyellow and Vgreen are calculated segment by segment. Since different segments have different traffic conditions, each segment has its own Vyellow and Vgreen. Therefore, we can use Vyellow and Vgreen to classify Vmax into three levels as follows:
(1) Red: Vmax < Vyellow,
(2) Yellow: Vyellow ≤ Vmax < Vgreen,
(3) Green: Vgreen ≤ Vmax.
Table 4-5 An example of selection of Vyellow and Vgreen. Vyellow: 36, Vgreen: 41
Vavg Level Vmax Level
29 G 42 G
28 G 41 G
25 G 42 G
25 Y 43 G
22 Y 41 G
23 Y 40 Y
23 Y 41 G
23 Y 41 G
21 Y 40 Y
22 Y 36 Y
Overall accuracy(True Positive): 60 % Table 4-6 An example of adjustment.
Vyellow: 36, Vgreen: 42
34
Table 4-7 shows an example of classification using Vavg. The traffic flow in this example is set to be light and stable. We can observe that these data fluctuate periodically due to the effect of traffic lights, i.e., red level occurs at time 510 and 630.
However, each traffic state is supposed to be green level because of stable traffic.
Since these data fluctuate and therefore generate incoherent information, we smooth them out by using simple moving average (SMA) algorithm. As shown in Table 4-8, these data are averaged with previous 4 samples. As a result, these data generate understandable traffic information after smoothed out.
Table 4-7 An example of classification.
Time Vavg (kph) Level
450 38 G
480 37 G
510 13 R
540 38 G
35
570 41 G
600 34 G
630 12 R
Table 4-8 An example of classification after smoothing out the data.
Time Vavg (kph) Level
450 35 G
480 35 G
510 32 G
540 32 G
570 32 G
600 32 G
630 31 G
After determine the classification rule successfully, we validate the accuracy of data derived from Vmax in terms of LOS. A result is considered correct or true positive when the traffic state derived from Vmax matches with the one derived from Vavg. The accuracy results of data derived from Vmax are shown in Table 4-9. The evaluation results show that true positive rates range from 0.700 to 1.000, false positive rates range from 0.000 to 0.104, and the average accuracy achieves 94.79%. Consequently, the findings indicate that Vmax provides highly accurate LOS.
Table 4-9 The accuracy of data derived from Vmax. a. Segment 1: 615 m Vyellow: 31, Vgreen: 39
Level Occurrences True Positive rate False Positive rate
Green 43 1.000 0.104
Yellow 17 0.765 0.039
Red 60 0.867 0.000
36
Overall accuracy (True Positive): 90.00 % b. Segment 2: 500 m Vyellow: 35, Vgreen: 40
Level Occurrences True Positive rate False Positive rate
Green 30 1.000 0.000
Yellow 8 1.000 0.000
Red 82 1.000 0.000
Overall accuracy (True Positive): 100.00 % c. Segment 3: 461 m Vyellow: 36, Vgreen: 42
Level Occurrences True Positive rate False Positive rate
Green 10 0.800 0.055
Yellow 20 0.700 0.020
Red 90 1.000 0.000
Overall accuracy (True Positive): 93.33 % d. Segment 4: 740 m Vyellow: 31, Vgreen: 40
Level Occurrences True Positive rate False Positive rate
Green 7 1.000 0.018
Yellow 64 0.969 0.054
Red 49 0.939 0.000
Overall accuracy (True Positive): 95.83 %
4.3 The Minimum Travel Time
To show that our system is able to provide intuitive information of traffic trends to road users, we compared the traffic information system we developed with the traditional one using average travel time. In the case of obtaining average travel time, each probe car reports its travel time when it reaches at the end of segment. The TIC averages received data during each period. To analysis time trends, the evaluated length is 2400 meters in this experiment. In order to find out time trends, travel time data are also smoothed out by using SMA algorithm. In other words, these data are
37
averaged with previous 4 samples. The time difference between each sample is 30 seconds, which is the same as the broadcast interval.
Fig. 4-2 depicts the comparison of time trends between the average travel time (TTavg) and the minimum travel time (TTmin). The MAE between these two kinds of travel time data was 18.9, which was small. The results indicate that our system is capable of providing reliable traffic information to road users.
Fig. 4-3 is a scatter diagram which illustrates the relationships between TTavg and TTmin. The correlation coefficient between them was 0.9987, which was very high.
The results show that TTmin is highly correlates with TTavg. Therefore, it is possible to use TTmin as a traffic indicator instead of using TTavg.
Fig. 4-2 Comparison of time trends between TTavg and TTmin.
38
Fig. 4-3 Correlation between TTavg and TTmin.
0 100 200 300 400 500 600 700 800 900
0 200 400 600 800 1000
Mminimum Travel Time [second]
Average Travel time [second]
39