Real-Time Identification

Assume we finish the S²DP algorithm that all vehicles in the second camera are assigned. Then, if a new vehicle shows up in the second camera, we can assign a candidate to this vehicle in real time. Instead of considering a group of vehicles from two cameras in S²DP, the Real-Time (RT) algorithm only takes one vehicle in the latter camera for identification. RT does not have to wait for a number of vehicles passing by, thus achieve real-time processing. This section describes a simple greedy algorithm that achieves real-time processing of the identification problem.

Given the computation result of the S²DP and a feature vector of one newly detected vehicle in the second camera, the RT assigns one candidate vehicle to the newly detected one. As we finish the initialization step using S²DP, a newly detected

34

vehicle in the second camera is possibly next to the last assigned one in the first camera. This property suggests us of using a sliding window approach to solve the problem.

The Real-Time (RT) algorithm runs as follows. First set a search window in the detection queue of the first camera just around the last assigned vehicle in the S²DP, since a newly detected vehicle in latter camera is probably behind it according to the ordering constraint. Considering the example in Figure 3.8, the S²DP matches vehicle A to vehicle D. As vehicle E is detected in camera C₂, the desired matching candidate in C1 is probably around vehicle D. The size of the window is limited to a small value for real-time processing, for example, five vehicles in our experiments. Notice that the detection queue of the first camera always contains unassigned vehicles since it is prior than the second camera. If a vehicle appears in the second camera, we calculate the feature vector of the detection and compute the distance to all vehicles in the search window of the first camera. The one in the first camera with the smallest distance is picked up and treated as the assignment result. If all distance values are greater than the matching threshold, which is greater than 2𝜖 in S²DP, a no-match is assigned. Finally we slide the search window to the next one in the detection queue of the first camera, and this process loops again. Figure 3-13 describes the RT algorithm in detail. As an execution example shown in Figure 3-14, only five candidates in C1

are considered when vehicle E is detected in camera C₂. The candidate with smallest distance is matched. Next five candidates are changed using sliding window approach for the next detected vehicle F in C₂. Note that only the candidates in the search window are taken into consideration. The distance values of others, for example, candidate J, are not computed and are represented using dots.

35

Figure 3-13. The Real-Time (RT) algorithm.

Figure 3-14. An example of RT. Each capital letter represents one vehicle, and the table is the distance matrix. Only the distance values of candidates are computed and

presented using numbers.

The RT algorithm allows one candidate in the first camera being matched to two or more new detections in the second camera. It is strange and erroneous that one vehicle became two in another camera. However in our experiments, we found that error may propagate if we do not enforce the property and produces poor identification results. Consider the scenario in Figure 3-15 that vehicle X in the first

• Assume 𝑁 candidates in the search window of the first camera

• For each newly detected vehicle in the second camera – Calculates the distances to 𝑁 candidates

– Picks up the candidate with the smallest distance value – Outputs the selected one if the distance is less than threshold

• Otherwise assign a no-match

– Shifts the search window in the first camera

36

camera C₁ is assigned to vehicle W in the second camera C₂, which is an incorrect assignment. The algorithm would delete X from the candidate queue if we do not allow re-assignment. Next vehicle X in C₂ was detected, however X in the candidate queue had already been erased and the assignment would certainly incorrect. If X in C₂ still got another false assignment, then this error propagated. Therefore an incorrect assignment will possibly produce at least two errors in the final result.

Figure 3-15. An example of error propagation if popping-candidate is allowed. In the beginning vehicle W is detected in C₂ and erroneous matched vehicle X in C₁. Next

vehicle X is detected in C₂, but X has already been removed from candidates thus cannot obtain correct result.

Another characteristic is that the last assignment in the first camera has to be a correct assignment. Otherwise the initial position of the sliding window would be inappropriate and the correct candidate would never appear in the search window.

Hence the whole real-time identification results would be wrong. To make sure the algorithm can correctly find the location of the sliding window, we discard the last three results in the S²DP and re-assign them in the RT. Using the S²DP algorithm can

37

make all assigned candidates close to each other by considering the NS² penalty.

However, the vehicles in the end of the matched group cannot be matched correctly since the last few vehicles of the group have too little information to be matched.

Consequently, we discard the last three assignments and re-assign them in the RT.

Finally, to enhance the effect of ordering constraint 𝜆 ∗ 𝑠𝑡𝑒𝑝 is added to all distance values, where 𝜆 is the NS² penalty and 𝑠𝑡𝑒𝑝 is number of steps from previous assignment.