We developed and implemented of an video segmentation system on the personal com-puter.
The core of our system was the graph-based edge linking technique. We used the change detection technique to get the object mask roughly. For easier obtaining the rel-ative thresholds of each module a two staged method for camera noise estimation was introduced to reduce the effect of moving objects and those thresholds were adjusted based on the estimated camera noise. We used the Canny operation to detect the edge of entire frame and we removed the edges of background by the change detection mask and the object mask of previous frame. Then we shrinked the object mask to the edge map by roll operation and we could get the object mask. To refine the object mask, we employed edge-link method, postprocessing, and temporal filter which was based on the background registration technique.
If the format of video sequence was QCIF (176 × 144), the proposed segmentation method could achieve 20 frames per second. For CIF format application, we also pro-posed a simple segmentation method. The frame rate could achieve 12 frames per second.
Simulation results showed that our algorithm could give correct segmentation results very quickly. According to these features, our algorithm was very suitable for videophone and videoconference with stationary background.
For quality improvement we can do some improvements for the main projects in the future.
1. Adding the module to deal with shadow and light change.
The position of shadow is controlled by the position of light and therefore the shadow effect greatly depends on the position of light. If the great shadows appear in background, the shadows are also regarded as moving objects in the module of background subtraction. Hence, a module to reduce the shadow effect can improve the accuracy of final image mask when the shadow is great.
2. Combine the segmentation system with MPEG-4 encoder.
The MPEG-4 Encoder is completed by another member in our laboratory. In the future, we will combine the segmentation system and MPEG-4 encoder to achieve a whole system.
3. More optimization.
The current processing time is not fast enough for CIF format and a better opti-mization is need to improve the efficiency. The optiopti-mization may focus on labeling, edgelink, and SetPixel functions.
Bibliography
[1] J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 6, pp. 679–698, Nov. 1986.
[2] D. W. Dijkstra, “A note on two problems in connexion with graphs,” Numer. Math., vol. 1, pp. 269–271, 1959.
[3] T. Aach, A. Kaup ,and R. Mester, “Statistical model-based change detection in mov-ing video,” Signal Processmov-ing, vol. 31, pp. 165–180, Mar. 1993.
[4] S. Y. Ma, S. Y. Chien, and L. G. Chen, “Efficient moving object segmentation al-gorithm using background registration technique,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 7, pp. 577–586, July 2002.
[5] T. Meier and K. N. Ngan, “Video segmentation for content-based coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 8, pp. 1190–1203, Dec. 1999.
[6] E. P. Ong, B. J. Tye, W. S. Lin, and M. Etoh, “An efficient video object segmentation scheme,” in IEEE Int. Conf. Acoust. Speech Signal Processing, vol. 4, 2002, pp.
3361–3364.
[7] H. Luo and A. Eleftheriadis, “Rubberband: an improved graph search algorithm for interactive object segmentation,” in IEEE Int. Conf. Image Processing, vol. 1, 2002, pp. 101–104.
[8] Y-H. Lin, “Real-time video segmentation based on background modeling for video-conferencing,” M.S. thesis, Department of Electronics Engineering, National Chiao Tung University, Hsinchu, Taiwan, R. O. C., June 2004.
[9] R. C. Gonzales and R. E.Woods, Digital Image Processing. Addison-Wesley, 1992.
[10] C. Kim and J. N. Hwang, “Fast and automatic video object segmentation and track-ing for content-based application,” IEEE Trans. Circuits Syst. Video Technol., vol.
12, no. 2, pp. 122–129, Feb. 2002.
[11] Y-H. Jan and D. W. Lin, “Video segmentation with extraction of overlaid objects via multi-tier spatio-temporal analysis,” Int. J. Electrical Engineering, vol. 11, no.
3, pp. 205–218, Aug. 2004.
[12] “Canny operator code,” http://ouray.cudenver.edu/ na0alber/DataCompressionPaper.htm.
[13] Intel, Getting Started With the VTune (TM) Performance Analyzer. 2003.
[14] “Video for Windows,” http://msdn.microsoft.com/library/psdk/multimed/avifile 8dgz.htm.
[15] “VidCap: full-featured video capture application,”
http://msdn.microsoft.com/library/devprods/vs6/visualc/vcsample/vcsmpvidcap.htm.
[16] Intel, MMX Technology — Programmers Reference Manual 2000.
[17] Intel, IA-32 Intel Architecture Software Developer’s Manual, vol. 1. 2003.
[18] Intel, IA-32 Intel Architecture Software Developer’s Manual, vol. 2. 2003.
[19] M-Y. Liu, “Real-time implementation of MPEG-4 video encoder using SIMD-enhanced Intel Processor,” M.S. thesis, Degree Program of Electrical Engineering and Computer Science, National Chiao Tung University, Hsinchu, Taiwan, R. O. C., July 2004
[20] M. Wollborn and R. Mech, “Refined procedure for objective evaluation of VOP gen-eration algorithms,” Doc. ISO/IEC JTC1/SC29/WG11 MPEG98/3448, Mar. 1998 [21] T. Meier and K. N. Ngan, “Automatic segmentation of moving objects for video
object plane generation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, pp. 525–
538, Sep. 1998.
[22] E. Sifakis, L. Grinias, and G. Tziritas, “Video segmentation using fast marching and region growing algorithms,” EURASIP J. Applied Siginal Processing, vol. 8, pp.
379–388, Apr. 2002.
[23] S. Y. Chien, Y. W. Huang, and L. G. Chen, “Predictive Watershed: A Fast Watershed algorithm for video segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol.
13, no. 5, pp. 453–461, May 2003.
[24] W. Si, Z. Yong-dong, and L. Shou-xun, “An automatic segmentation algorithm for moving objects in video sequence under multi-constraints,” in IEEE Int. Conf. Mul-timedia Expo, vol. 1, June 2004, pp. 555–558.
[25] W. Wei, K. N. Ngan, and N. Habili, “Multiple feature clustering algorithm for automatic video segmentation,” in IEEE Int. Conf. Accoustics Speech and Signal Processing, vol. 3, May 2004, pp. 625–628.
[26] “Demo page of video segmentation,” http://video.ee.ntu.edu.tw/web/demo/seg.htm.