國
立
交
通
大
學
資訊科學與工程研究所
博
士
論
文
視覺式車輛特徵分析之研究
Research on Vision Based Car Feature Analysis
研 究 生：古蕙媜
指導教授：李素瑛 教授
視覺式車輛特徵分析之研究
學生：古蕙媜 指導教授：李素瑛 博士
國立交通大學資訊科學與工程研究所
摘要
由於交通監控系統的普及化，偵測、追蹤並且辨識道路影片中的移動物體， 逐漸成為一項重要的研究議題。由於大部份的事故是由汽車所引起的，而且車輛 的外型特徵相較於車牌號碼，更不易被偽造或者隱藏，因此本論文將建構一個智 慧型交通監控系統，能夠在各種攝影機角度、各種反光影響下，辨識車輛的外型 特徵，包括顏色、大小與款式。為了克服光影的變化，排除車窗、車燈等無關車 色像素的影響，本論文提出一個三階段車體切割演算法，分別針對亮色系、暗色 系與彩色系的車子，設計不同策略切割車體，車輛的顏色僅考慮車體內部像素， 因此可以得到更精準的車輛主顏色，更進一步得到更正確的車色分類結果。為了 快速地估計車輛大小以及旋轉角度，本系統提出對稱中心演算法，用來搜尋車頭 的對稱中心，並計算此中心到最近邊緣的距離，此距離與車輛高度以及寬度的比 例，分別與車輛的大小與角度呈現單調函數，因此車輛的大小與角度可藉由比例 的反函數來求得。為了在各種角度下更精準的辨識車款，我們提出鏡射形變技 術，此技術可將在各種角度下拍攝到的車子都調整成標準的正面、側面或者背 面，接著選擇與測試車相同角度，並同樣經過形變處理的樣本車來比對。由於鏡 射形變技術可有效地排除角度估計誤差以對稱中心搜尋誤差，因此能比傳統以角 度估計為基礎的車款辨識方法提供更高的正確率。 關鍵詞：車輛顏色識別、車輛大小分類、車輛角度估計、車輛款式辨識、智慧型 交通系統、三階段車體切割、鏡射形變Research on Vision Based Car Feature Analysis
Student: Hui-Zhen Gu Advisor: Dr. Suh-Yin Lee
Department of Computer Science and Information Engineering
National Chiao Tung University
Abstract
With the rapid growth of surveillance equipments, detecting, tracking, and recognizing
moving objects in roadway videos is currently a popular issue. Because most accidents are
caused by cars and the appearance features of cars are hard to be hidden or counterfeited like
license plate, this research develops an intelligent traffic monitoring system which identifies
appearance features, such as sizes, models, and colors under varying camera viewpoint and
light reflections. Due to the effect of non-homogeneous light reflection, the improper
foreground pixels, such as the windshield, or lamps influence the extraction of color type. A
tri-states car body segmentation algorithm is proposed in this dissertation. Different strategies
are designed for bright, dark, and colored cars, and only the pixels belonging to the car body
are considered for color classification. Therefore, a purer car color can be extracted and a
more correct color type can be classified. To rapidly estimate the size and pose of a car, a
symmetric center detection algorithm is proposed. The algorithm searches the symmetric
center on the head (or rear) of a car and computes the distance between the center and the
closest boundary as half of the head width. Two aspect ratios: car height to head (or rear)
To recognize car model across varying poses, a mirror morphing scheme is proposed. The
scheme is able to transform cars with varying poses into a typical (front, rear, or side) view.
Then a template car with the same pose with the tested car is selected and matched against the
tested car. Because the mirror morphing scheme effectively reduces center bias and estimation
error of tested and template cars, higher recognition rate can be anticipated. Finally, the
experiments show that the proposed system is superior to conventional approaches for
classifying colors, sizes, and models of cars.
Keywords：car color determination, car pose estimation, car size classification, car model
recognition, intelligent transportation system, tri-states car body segmentation,
Table of Contents
摘要……… i
Abstract………⋯ ii
Table of Contents……… iv
List of Figures……… vi
List of Tables……… viii
誌謝……… ix
Chapter 1 Introduction ……… 1
1.1 Research motivation……… 1
1.2 Research objective……… 3
1.3 Proposed solutions……… 4
Chapter 2 Related Works……… 7
2.1 Current research on car color determination……… 7
2.2 Current research on car pose estimation……… 9
2.3 Current research on car size classification……… 10
2.4 Current research on car model recognition……… 12
Chapter 3 Car Color Determination Algorithm……… 14
3.1 Algorithm overview……… 15
3.2 Car body candidate generation……… 17
3.3 Car body determination……… 35
3.4 Car color classification……… 37
3.5 Experiments of car color determination……… 38
Chapter 4 Car Pose Estimation Algorithm……… 53
4.1 Pose ratio computing……… 53
4.2 Symmetric center detection……… 56
4.3 Experiments of car pose estimation……… 61
Chapter 5 Car Size Classification Algorithm……… 64
5.1 Concavity detection……… 64
5.3 Three-class support vector machine……… 67
5.4 Experiments of car size classification……… 68
Chapter 6 Car Model Recognition Algorithm……… 71
6.1 Shape feature extraction……… 71
6.2 General morphing……… 74
6.3 Mirror morphing……… 76
6.4 Template matching……… 79
6.5 Experiments of car model recognition……… 82
Chapter 7 Conclusions and Future Works……… 97
7.1 Conclusions……… 97
7.2 Future works……… 98
List of Figures
Fig. 1: (a) Some examples of black, gray, white, red, yellow, green, and blue cars, (b) small-size cars, medium-size cars, and big-size cars, (c) car models, including
Mitsubishi lancer, Mazda 3, Nissan march, Honda civic, and Toyota yaris……… 2
Fig. 2: Car images with non-homogenous light reflection……… 3
Fig. 3: Car images covering different vehicle orientations……… 3
Fig. 4: System overview……… 4
Fig. 5: The car body extraction and color classification framework……… 14
Fig. 6: White, black, and orange car images and their segmentation results by hill-climbing algorithm……… 18
Fig. 7: (a)-(c) The specular-free images of the white, black, and orange cars, (d)-(f) the segmented results of the specular-free images via hill-climbing algorithm……… 20
Fig. 8: The extracted car body candidates of the (a) white, (b) black, and (c) orange cars after the process of the SARM algorithm……… 26
Fig. 9: The intensity (a)-(d) and saturation (e)-(f) histograms of six cars with black, white, dark-gray, light-gray, red and yellow colors and the background………… 30
Fig. 10: The CovAcc analysis with various BinNum and MergNum parameters for (a) black, (b) white, (c) dark-gray, (d) light-gray, and (e) colored cars. (f) Optimal parameters table……… 34
Fig. 11: Hierarchical SVM of color type classification……… 38
Fig. 12: Covering precisions of three compared methods on (a) ITS, (b) CBIR-simple, and (c) CBIR-complex datasets for each color category. Covering recalls of three compared methods on (d) ITS, (e) CBIR-simple, and (f) CBIR-complex datasets for each color category……… 44
Fig. 13: Accumulated percentage of hue deviation by three compared methods on colored car images in (a) ITS, (b) CBIR-simple and (c) CBIR-complex datasets. Accumulated percentage of intensity deviation by three compared methods on grayscale car images in (d) ITS, (e) CBIR-simple, and (f) CBIR-complex datasets 47 Fig. 14: Accuracies of color type classification of the three compared methods on in (a) ITS, (b) CBIR-simple, and (c) CBIR-complex datasets……… 50
Fig. 15: (a) The left, right, top, bottom ends and the symmetric center of a car, (b) the ratio cov of car width to car height and its estimation error cov (error bar), (c) the ratio pos of half width to car width and its estimation error pos……… 54
the check width and the check point at x-axis = 27, (c) the check point at 43 and the pixels in correspondence to compute ASD……… Fig. 17: ASD results and the prominent factor results for all check points……… 58 Fig. 18: The estimated pose errors of the conventional approach adopting cov and the
proposed approach adopting pose on (a) theoretical analysis and (b) the
experimental results……… 62
Fig. 19: Flowchart of the proposed size classification algorithm……… 64 Fig. 20: The concavity triangles appeared in a (a) small and (b) big car with large poses… 65 Fig. 21: Point distribution of (rsize, rpose) of small, medium, and big cars across various
poses……… 67
Fig. 22: ASM result (a) with erroneous initial position, (b) with improper initial mean shape, (c) with proper initialization setting. (d) ASM result of a different model car, (e) the segmented result of (c), and (f) the segmented result of (d)………… 73 Fig. 23: Establish the corresponding relationship between Img1 and Img2 for all pixels
based on known key point pairs (a) source image Img1 with 61 key points (b)
source image Img2 with 61 key points (c) a pixel S in Img1 but not one of 61 key
points (d) the pixel S’ in Img2 corresponding to the pixel S in Img1……… 75
Fig. 24: (a) Segmented image of front region, (b) mirror image of (a), (c) synthesized car image from (a) and (b) by mirror morphing, (d) synthesized template car image, (e) segmented image of side region, (f) selected mirror template, (g) synthesized car image from (e) and (f) by quasi mirror morphing……… 78 Fig. 25: Sensitivities of the algorithms with and without mirror morphing to pose error
and center bias in the initial phase……… 81 Fig. 26: Accuracy of estimated pose angle over all orientations……… 89 Fig. 27: Accuracy of estimated center position over all orientations……… 89 Fig. 28: Matching error distributions of (a) PETM-A, (b) PETM-B, (c) PASM-A, (d)
PASM-B, (e) ASMM-A, (f) ASMM-B, (g) QRAFM adopting the template car whose orientation is 30 ﾟ, and (h) QRAFM adopting 50 ﾟ template on target cars and non-target cars……… 92 Fig. 29: Recognition rates over all orientations of the PETM, PASM and ASMM
algorithms……… 93
Fig. 30: Recognition rates over all orientations of the ASMM-B, PE-QRAFM , QRAFM with 45 ﾟ and 135 ﾟ templates, and PASM-B algorithms……… 95
List of Tables
Table 1: Terminology table of the car color determination algorithm……… 15
Table 2: The generated CBRs for the white, orange, and black cars in Fig. 6(a)-(c)…… 36
Table 3: Sample images from the ITS dataset……… 40
Table 4: Sample images from the CBIR-simple and CBIR-complex dataset……… 41
Table 5: Confusion matrices of color category classification on ITS, CBIR-simple, and CBIR-complex datasets……… 43
Table 6: Average computation time of each component in the proposed car color determination algorithm and in the compared methods……… 52
Table 7: Accuracy of the concavity detection……… 68
Table 8: Accuracy of the symmetric center detection……… 69
Table 9: Accuracy of the size classification……… 70
Table 10: The functions working in the car model recognition algorithms……… 84
Table 11: Examples of database images. Each row displays car images in the same view. The first car with red boundary is the target model and the others are non-target models……… 85 Table 12: Average computation time of each step in the car model recognition algorithm 96
![TraditionalMLCalgorithmsmainlytacklethebatchMLCproblem,wheretheinputdataarepresentedinabatch[24,28].Nevertheless,inmanyMLCapplicationssuchase-mailcategorization[22],multi-labelexamplesarriveasastream.Onlineanalysisistherefore dimensionreducermotivatedbyma](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)