Chapter 3 Background Modeling in HSV and CIELAB Color Space
D. Color Compensation
4.2 Foreground subjects extraction
4.2.1 Foreground Detection in HSV Color Space
In segmenting the images, the V color component is usually stable and reliable, but it has two drawbacks: the V component is insensitive to the similar, especially lighting, color such as yellow, pink, and light blue. When the subjects wear the clothing with the color different from the background, we can do background subtraction well in the V color component.
In the first step, we use the frame ration in the V color component to get the binary image B x y in Eq. (24) described in Sec. 3.1.3. The value ( , ) k is chosen by V
experiments and varies with different trials. Hence, we ran a series of experiments to determine the optimal threshold k When the subject’s clothing color different from V. the background, Fig. 4.7 shows the binary image ( , )B x y obtained by different
V .
k s＇ When subject’s clothing color similar to the background, Fig. 4.8 shows the binary image B x y obtained by different ( , ) k sV＇. Comparing Figs. 4.7 and 4.8, we can find that if the color is different from the background, we can use the threshold value k to get a good foreground subject extraction. But we cannot adjust V k to V
get a complete and noise-free foreground subject when the clothing color is similar to the background. After the experiment, we set kV =1.3 in the HSV color system.
(a) (b)
(c) (d)
(e) (f)
Fig. 4.7. An example of foreground extraction at different k thresholds. V
(a) An image frame with subject’s clothing color different from the background, (b)−(f) foreground detected images, (b) kV =1.0, (c) kV =1.1, (d) kV =1.2, (e)
V 1.3
k = , and (f) kV =1.4
(a) (b)
(c) (d)
(e) (f)
Fig. 4.8. An example of foreground region extraction at different k threshold. V
(a) An image frame with subject’s clothing color similar to the background, (b)−(f) foreground detected images, (b) kV =1.0, (c) kV =1.1, (d) kV =1.2, (e) kV =1.3, and (f) kV =1.4,
During the foreground extraction, the shadowing effect introduces artifact foreground subjects and deteriorates the recognition result. We use the shadow mask, which including the shadow characteristic existing in HSV domains of Eq. (25) described in Sec. 3.1.4 to classify the pixels whether it is a shadow point or not. Fig.
4.9 shows the process result regarding shadow suppression. Figs. 4.9(a) and (b) are two input images. Figs. 4.9(c) and (d) are the foreground subject without shadow suppression. The foreground subject with shadow suppression is shown in Figs. 4.9(e) and (f), which improves greatly comparing with Figs. 4.9(c) and (d).
(a) (b)
(c) (d)
(e) (f)
Fig. 4.9. The example of the shadow suppression.
The models wear light blue clothing, yellow clothing, and pink clothing, respectively. In the previous experiment, we cannot adjust k to get a complete and V clean foreground subject. Hence, we do the color compensation in Eq. (26) described in Sec. 3.1.3. In what follows, the effectiveness of color compensation in obtaining a more accurate foreground is described in Fig. 4.10.
From the Figs. 4.10(a2)-(c2), we can find a trade-off between the foreground and the background detection by color compensation step to the whole image. Hence we cannot get a complete and noise-free foreground subject when the clothing color is similar to the background.
From the Figs. 4.10(a3)-(c3), we have found that we can get good compensation when the clothing color is light blue and yellow, but cannot obtain good compensation when the clothing color is pink. The reason is that when pink color pixels are transformed from RGB color space to HSV color space, the saturation of pink is lower than the set criterion S . Hence, we cannot recover those pixels from t
background to foreground for such small chromaticity difference in this space.
(a) (a1)
(a2) (a3)
(b) (b1)
(b2) (b3)
(c) (c1)
(c2) (c3)
Fig. 4.10. Foreground detection without and with color compensation. (a)−(c) is the input images, (a1)−(c1) the foreground images, without color compensation, (a2)−(c2) the foreground images detected with color compensation to the whole image. (a3)−(c3) the foreground images detected with color compensation to only foreground subject region.
4.2.2 Foreground Detection in CIELAB Color Space
We utilize the maximum inter-frame color difference d(x, y) of the training background model to get the binary image B x y in Eq. (32) described in Sec. 3.2.3, ( , ) and use the “foreground subject ground truths” to record the color difference of foreground pixel simultaneously. Fig. 4.11 shows the histogram of color difference of the foreground subject.
(a) (b) (c)
Fig. 4.11. The histogram of color difference of the foreground subject. (a) the clothing color is light pink, (b) the clothing color is light yellow and (c) the clothing color is light blue.
The value k is chosen by experiments and varies with different trials. Hence, we ran a series of experiments to determine the optimal threshold k. When subject’s clothing color different from the background, Fig. 4.12 shows the binary image
( , )
B x y obtained by different k’s. When subject’s clothing color similar to the background, Fig. 4.13 shows the binary image ( , )B x y obtained by different k’s.
(a) (b)
(c) (d)
(e) (f)
Fig. 4.12. An example of foreground extraction at different k thresholds.(a) An image frame with subject’s clothing color different from the background, (b)−(f) foreground detected images, (b) k = 1, (c) k = 2, (d) k = 3, (e) k = 4, and (f) k = 5
(a) (b)
(c) (d)
(e) (f)
Fig. 4.13. An example of foreground extraction at different k thresholds.(a) An image frame with subject’s clothing color similar to the background, (b)−(f) foreground detected images, (b) k = 1, (c) k = 2, (d) k = 3, (e) k = 4, and (f) k = 5
From Fig. 4.12 and Fig. 4.13, we can find that if the color is different from or similar to the background, we can use the threshold value k to get a good foreground subject extraction in the CIELAB space. In general condition, the suitable range of
4.3 Comparing the Experimental Result
The results of the foreground subject extraction in the HSV and CIELAB color spaces are showed in Fig. 4.14, the left column contains input images; the middle column contains the resulting foreground images detected in the HSV color space;
and the right column is the resulting foreground images detected in the CIELAB color space.
(a) (a1) (a2)
(b) (b1) (b2)
(c) (c1) (c2)
(d) (d1) (d2)
(e) (e1) (e2)
Fig. 4.14 The result of the foreground subject extraction in the HSV and CIELAB color space. (a)−(e) is the input images, (a1)−(e1) the foreground images detected in the HSV color space, (a2)−(e2) the foreground images detected in the CIELAB color space.
We selected over 300 frames from the video sequence of the model with a subject wearing clothing similar to the background color. The “foreground subject ground truths” of these 300 frames were generated manually. Let A be a detected foreground subject region and B be the corresponding “ground truth.” Then we test the pixel accuracy by the following two metrics. Metric 1, accuracy rate , is a 1 measure concerning whole segmented region pixels relative to these pixels in A the same with in B. To this end, we calculate the accuracy rate by
where Ntotal is the pixel number of segmented foreground image, and N is the s
pixel number that the pixel in A is the same as that in B, i.e., such of true positive and false negative pixels of A relative to B. Metric 2, accuracy rate , is adopted from [21] 2 by
Accuracy rate2 A B 100%.
A B
= ∩ ×
∪ (38)
This measure counts the percentage of the mutual positive pixels to expanded positive pixels. We consider the accuracy rate of the foreground subject and the background in metric 1 and 2. Table I and III show the accuracy rate of the foreground subject and the background in metric 1 and 2 of over 300 frames, and the HSV (i) and (ii) is the accuracy rate of the foreground images detected with color compensation to the whole image and only foreground subject region, respectively. Table II and IV show the combination accuracy rate of the foreground subject combined with the background by linear interpolation, and demonstrate the improvement of the foreground subject extraction in the CIELAB color space over that in the HSV color Space.
TABLE I
COMPARISON RESULT OF THE PIXEL ACCURACY RATES OVER 300IMAGES INMETRIC1
Accuracy rate (%) 1
HSV (i) HSV (ii) CIELAB
Foreground Background Foreground Background Foreground Background
Pink 58.91 98.86 62.39 99.42 90.55 98.72
Yellow 82.72 96.32 82.06 99.26 90.67 98.64
Light
Blue 78.78 95.06 89.82 99.58 92.96 98.67
White 67.33 98.03 71.68 98.91 83.24 99.15
Ivory 58.42 98.31 64.21 99.13 77.55 99.35
TABLE II
THECOMBINATIONACCURACYRATES INMETRIC1
Combination Accuracy rate (%) 1
HSV (i) HSV (ii) CIELAB
Pink 60.24 64.46 91.02
Yellow 83.5 83.02 91.44
Light Blue 81.41 90.1 93.24
White 69.53 73.72 84.47
Ivory 60.14 66.82 79.21
TABLE III
COMPARISON RESULT OF THE PIXEL ACCURACY RATES OVER 300IMAGES INMETRIC2
Accuracy rate (%) 2
HSV (i) HSV (ii) CIELAB
Foreground Background Foreground Background Foreground Background
Pink 48.47 96.42 56.69 97.22 86.41 98.37
Yellow 52.27 95.32 73.09 98.22 85.41 98.15
Light
Blue 42.59 93.96 83.14 99.02 87.73 98.35
White 51.32 96.77 63.03 96.69 75.11 97.78
Ivory 49.17 96.01 57.89 96.35 71.67 97.56
TABLE IV
THECOMBINATIONACCURACYRATES INMETRIC2
Combination Accuracy rate (%) 2
HSV (i) HSV (ii) CIELAB
Pink 51.17 58.97 87.62
Yellow 54.68 74.49 86.12
Light Blue 45.26 83.96 88.28
White 53.02 65.54 76.82
Ivory 41.45 60.76 73.64
Average 49.12 68.74 82.5
Chapter 5 Conclusion
In this thesis, we have proposed the foreground subject extraction in the CIELAB color space. Embedded in CIELAB space, our method exploits color difference formula to raise the sensitivity of color detection. In the CIELAB color space, we still can utilize not only the luminance component but also the chromatic component existent in the background image. In this way, we can reliably extract the foreground subject, even when the foreground chrominance is similar to that of the background. Experimental results have shown of the foreground subject extraction is better in the CIELAB color space than HSV Color space.
In the future study, we can apply our method to human activity recognition system. The recognition rate can be raised owing to better segmentation capability. In addition, utilization other color difference formulae, detection by a camera moving at a fixed velocity, extensions of various test environments, and more complicated surrounding are our future work.
References
[1] J. Yamato, J. Ohya, and K. Ishii, “Recognizing human action in time- sequential images using Hidden Markov model,” In Proc. IEEE CVPR, pp. 379−385, 1992.
[2] F. Bobick and J. W. Davis, “The recognition of human movement using temporal templates,” IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 3, 2001.
[3] I. Cohen and H. Li, “Inference of human postures by classification of 3D human body shape,＂ in Proc. IEEE Int. Workshop on Anal. Modeling of Faces and Gestures, pp. 74−81, 2003.
[4] I. Haritaoglu, D. Harwood, and L. S. Davis, “W4: real-time surveillance of people and their activities,” IEEE Trans. Pattern Anal. Machine Intell., vol. 22, no. 8, pp. 809−830, 2000.
[5] S. Park and J. K. Aggarwal, “Segmentation and tracking of Interacting human body parts under occlusion and shadowing,” in Proc. of the Workshop on Motion and Video Computing, pp.105−111, 2002.
[6] M. K. Leung and Y. H. Yang, “First sight: A human-body outline labeling system,” IEEE Trans. Pattern Anal. Machine Intell., vol. 17, no. 4, pp.
359−377,1995.
[7] S. Jabri, Z. Duric, H. Wechsler, and A. Rosenfeld, “Detection and location of people in video images using adaptive fusion of color and edge information,” in Proc. Int. Conf. Pattern Recognition, pp. 627−630, 2000.
[8] T. Horprasert, D. Harwood, and L.S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proc. IEEE ICCV’ 99, 1999.
[9] R. Cucchiara, C. Grana, M. Piccardi and A. Prati, “Improving shadow suppression in moving object detection with HSV color information,” in Proc.
IEEE Intelligent transportation System Conference, pp. 334−339, 2001.
[10] A. Prati, I. Mikic, M. Trivedi and R. Cucchiara, “detecting moving shadows:slgorithms and evaluation,” in Proc. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 918−923, 2003.
[11] B. Chen and Y. Lei, “Indoor and outdoor people detection and shadow suppression by exploiting HSV color information,” in Proc. Fourth Int. Conf. on Computer and Information Technology, pp. 137−142, 2004.
[12] S. Vitabile, G. Pilato, G. Pollaccia, and F. Sorbello, “Road signs recognition using a dynamic pixel aggregation technique in the HSV color space,” in Proc. 11th Int.
Conf. on Image Analysis and Processing, pp. 572−577, 2002.
[13] R. Cucchiara, M. Piccardi and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1337−1342, 2003.
[14] B. Hill, Th. Roger, and F. W. Corhagen, “Comparative analysis of the quantization of color space on the basis of the CIELAB color-difference formula,” ACM Transaction on Graphics, vol. 16, no. 2, April 1997, Pages 109-154.
[15] CIE. 1986a. Colorimetry. CIE Pub. 15.2, 2nd ed., Commission International de L’Eclairage, Vienna, 29-30
[16] Loo, M. R. and Rigg, B. 1987 BFD (1:c) colour-difference formula. Part 1:
Development of the formula. JSDC 103 (Feb.), 126-132. Part 2: Performance of the formula. JSDC 103 (March), 86-94.
Commission International de L’Eclairage, Vienna.
[19] CIE. 1995. Industrial color-difference evaluation. CIE Pub. 116, Commission International de L’Eclairage, Vienna.
[20] K. Ohba, Y. Sato, and K. Ikeuchi, “Appearance-based visual learning and object recognition with illumination invariance,” Machine Vision and Applications, Vol.
12, No. 4, pp. 189−196, 2000.
[21] L. Li, W. Huang, I. Y. H. Gu, and Q. Tian, “Statistical modeling of complex backgrounds for foreground object detection,” IEEE Transactions of Image Process, vol.13, no.11, pp.1459−1472, 2004.