Zoom control

The zoom in/out function will cooperate with the size selecting function of the tracking task. At the end of the tracking task, we need to determine if the size of target object become larger or getting smaller. As a matter of course, if the size of our target object becoming larger than some scale, we need control the camera to do the zoom out action. Otherwise, we do the zooming in action. For example in Figure 4-6, the size of the target object is too large, so we need to send the zoom out command. On the other hand in Figure 4-7, we need to perform the zoom in action.

Figure 4-6 : The target size is too large

Figure 4-7 : The target size is too small

The zoom in/out control method of this active camera is similar to the pan/tilt control method, but the difference is at the speed control of zoom. Figure 4.8 illustrates the controlling command of zoom-in, zoom-out and stop command. The difference between the Figure 4-8 and Figure 4-1 is that there is no speed control parameter fro the zoom control. The Figure 4-9 shows the flow chart of the function of zoom in/out control.

Zoom-In Zoom-out Stop 40 H 10 H FF H 40 H 20 H FF H 40 H 20 H FF H

Figure 4-8 : The Command Format of Zoom in/out

There is no speed control for zoom in or zoom out because its default setup, so we can only adjust the duration time of zoom in/out function to control the ratio of zoom. In Figure 4-9, we can see that after we sent a zoom in/out command, the camera will execute the zoom action until we send a stop command to terminate this action. Generally it takes 100 ms in every reasonable zoom period.

5 Chapter 5

Experimental Results

Because our moving object tracking system will run in real time video surveillance with an active pan-tilt-zoom camera, we should do some experiments to test its performance and stability under several kinds of environments. In this chapter, we will introduce the setup of our tracking system and define our experimental environment in section 5.1 and then compare our HSV color space to traditional RGB color space model in section 5.2.

In section 5.3, we will experiment our modified tracking method in real time tracking environment with an active PTZ camera and see the difference between our method and classical mean-shift tracking method. Finally, we will make a discussion in section 5.4.

5.1 The Experimental System

Our automatic tracking system is using the following components:

z The system uses Lilin PIH-7600 High Speed PTZ Dome active camera for capturing image sequence. The sequence is composed of 320x240 color image acquired at a frame rate 30 frames per second.

z The system is developed in Borland C++ Builder 6 and has been tested on color image sequences acquired on indoor environments.

z The system has been implemented on an Intel Pentium(R) 4 CPU running at 2.4 GHz with 512 Mb RAM and a real time video capture card under Windows XP SP2 OS.

z The active camera has two I/O interfaces which are RS-485 and video-out.

The RS-485 interface is used to drive the pan-tilt-zoom camera, but unfortunately, most PC does not support this specification (RS-485) so we use a RS-485 to RS-232 transfer connector to change our control signal format from RS-232 to RS-485. With respect to the video-out interface, it is an analog video output from the camera, so we can capture the real time video sequence through this video-out interface.

5.2 Environment Setup

The environment of our experimental locates in our laboratory. The complexity of the environment is enough to verify our system while tracking and detecting moving human. Figure 5-1, Figure 5-2 and Figure 5-3 show several images of our lab environment without zoom in/out operation. Figure 5-4, Figure 5-5 and Figure 5-6 show several images for zoom in/out condition.

Figure 5-1 : Experimental environment I

Figure 5-2 : Experimental environment II

Figure 5-3 : Experimental environment III

Figure 5-4 : Zoom condition I

Figure 5-5 : Zoom condition II

Figure 5-6 : Zoom condition III

The zoom in/out operation level of the pan-tilt-zoom camera which we use is analog controlled. We can control the zoom level by controlling the timing of stop command sending. For example, if we want to just zoom in only a little bit, we could send a stop command a short time, like 30ms, after sending the zoom in command. On the other hand, we could send a stop command a long time, like 300ms, after sending the zoom in command if we want to zoom in a lot. For control convenience, we separate the zoom in/out operation into about 9 stages for the overall system. Figure 5-7 shows a sequence of the total zoom operation.

Figure 5-7 : 9 stages of zoom in/out operation

5.3 Tracking Region Experiment

In our tracking system, we adopt the ecliptic tracking region rather than the rectangle one. We will do some experiments to prove the ecliptic tracking region which we use is better in both calculating loading and tracking performance. In this experiment, we will use a recorded video sequence because we need to confirm the all tracking situation variables are fixed and the only control variable is the tracking region. Here are our experiments:

I. We use the same video and the same original target object for the ecliptic tracking region or the rectangle tracking region to see which performance is more reliable, and show up in Figure 5-8 for the ecliptic tracking region and

in Figure 5-9 for the rectangle tracking region. We will find that the tracking failure will occur in Frame 432 by using rectangle tracking region.

Figure 5-8 : Tracking result using rectangle region

Figure 5-9 : Tracking result using ecliptic region

II. We try to analysis the iteration number of mean shift tracking system by comparing the rectangle tracking region, single layer weighting ecliptic tracking region and double layer weighting layer ecliptic tracking region. In

Figure 5-10, we can see the difference between these three tracking region and in Figure 5-11 we can find that the method of double layer weighting layer ecliptic tracking region has the best performance.

Figure 5-10 : Coordinate definitions and three different tracking regions

(X1, Y1) (X2, Y2) Rectangle Single Ecliptic Double Ecliptic

(97, 52) (152, 142) 2.14 1.99 1.63

Average Iteration Times 2.10 1.98 1.62

Figure 5-11 : Iteration time comparison (unit: times)

5.4 Color Space Experiment

In our system, we execute the transformation form the RGB color space to HSV in order to reduce the changing of the environment illumination having influence on our mean-shift tracking algorithm. Now, we will demonstrate the experimental result in Figure 5-12. In Figure 5-12, because the illumination of the target changes only a little bit, we can see that the tracking result of the mean-shift algorithm seems so far so good.

Figure 5-12 : Tracking with RGB color model

In Figure 5-13, when the illumination of the target changes a lot more, we can see that the tracking result of the mean-shift algorithm seems getting worse. In this case, we can find that the illumination influences the mean-shift tracking algorithm a lot, but this illumination effect is hard to find by human eyes actually. In this experiment, we can explain why we adapt the HSV color space rather than RGB color space. In Figure 5-13 shows up the result of using HSV color space.

RGB Tracking Result HSV Tracking Result RGB Tracking Result HSV Tracking Result

Figure 5-13 : Tracking with RGB and HSV color model at frame 391 and 412

5.5 Fix Size Tracking Experiment

In this experiment, we demo our tracking system which combines the video sequence analysis and automatic camera tracking control. Moreover, we adapt fix size tracking in this experiment, turn off the function zoom in/out and only use the automatic pan-tilt control for the active camera. In Figure 5-14 and Figure 5-15, we can see the result of this experiment and this tracking result will not influenced by the dynamic background.

Figure 5-14 : Fix size tracking result (sequence 1)

In Figure 5-15, we can find that although we do not choose the tracking region perfectly matched the target object in the reference frame, we still can track this target well under similar color background and many people interrupting in.

Figure 5-15 : Fix size tracking result (sequence 2)

In Figure 5-16, the result of single target tracking is demonstrated and the difference between Figure 5-16 and Figure 5-15 is the covering and disturbance becoming more complex and tough.

Figure 5-16 : Fix size tracking result (sequence 3)

5.6 Adaptive Size Tracking Experiment

In Figure 5-17, the result of adaptive size tracking is demonstrated and the major point of this section is the camera can not only automatically track target object by position, but also track target object by size, so we can keep the size almost the same by controlling the function of zoom in/out.

Figure 5-17 : Adaptive size tracking result

6 Chapter 6

Conclusions and Future Works

6.1 Conclusions

The experimental results show that the proposed system is capable of tracking moving object smoothly by an automatic controlled active camera. At the same time, the system also works well even if the target has some disturbances on its illumination and shape. There are several contributions made out of this research:

1. We provide a smooth camera tracking system.

2. Our system can locking on specific moving object, and does not be affected by illumination and background variables.

3. We proposed a modified mean-shift tracking algorithm, and that can be used for real time moving object tracking in both position and size selecting.

4. Our system can handling following conditions on tracking:

i. Camera acting

ii. Some illumination changing iii. Getting some covering noises

iv. Target object departing or closing to the camera

6.2 Future Works

So far, our tracking system works in indoor environment with one moving target although it still can do position tracking and size selecting, but when this target get some covering noises and become farer or closer to the camera at the same time, the proposed system can not track it well because we can not tell which situation occurs, covering disturbance or target size changing. In order to solve that problem, we must deal with the target object recognition problem and then we can know whether the target changes its size or it is covered by something else. The most suitable target object recognition system will be the human detection system for the indoor environment security surveillance system.

For setup convenience, this whole tracking algorithm can also be ported on some portable devices such as DSP platforms. That will be more suitable for real surveillance application and it also can be combined with more than one camera to enhance the tracking reliable intensity.

References

[1] Li, C. I., and Wang S. J., “Detection and Tracking of a Single Deformable Object on an Active Surveillance Camera,” 16th IPPR Conference on Computer Vision, Graphics and Image Processing (CVGIP 2003), 17~19 August, Kinmen, ROC [2] Segen, J. “A camera-based system for tracking people in real time,” Pattern

Recognition, 1996, Proceedings of the 13th International Conference, Volume 3, 25-29 Aug. 1996 Page(s):63 - 67 vol.3

[3] Daniilidis, K., Krauss, C., Hansen, M., and Sommer, G., “Real-Time Tracking of Moving Objects with an Active Camera,” Real-Time Imaging, Volume 4, Number 1, February 1998, pp. 3-20(18)

[4] Stauffer, C., and Grimson, W.E.L., “Adaptive background mixture models for real-time tracking,” Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on Volume 2, 23-25, June 1999

[5] Scheutz, M., McRaven J., and Cserey G.. “Fast, reliable, adaptive, bimodal people tracking for indoor environments,” in Proc. of the IEEE International Conference on Intelligent Robots and Systems, vol. 2, Oct. 2004, pp. 1347-1352.

[6] Comaniciu, D., Ramesh, V., and Meer, P., “Real-time tracking of non-rigid objects using mean shift,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC, volume II, June 2000, pp. 142–149.

[7] Po, L. M. and Ma, W. C., “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits System Video Technology, vol. 6, pp.

313-317, June 1996.

[8] Ghanbari, M., “The cross-search algorithm for motion estimation,” IEEE Trans.

Commun., vol. 38, pp. 950-953, July 1990.

[9] Shan, Z., and Ma, K. K. Ma, “A new diamond search algorithm for fast block

matching motion estimation,” Image Processing IEEE Transactions, page(s):

287-290, Feb 2000.

[10] Zhu, C., Lin, X., and Chau, L. P., “Hexagon-based search pattern for fast block motion estimation,” Circuits and Systems for Video Technology IEEE Transactions, page(s): 349-355, May 2002.

[11] Po, L. M., and Ma, W. C., “A novel four-step search algorithm for fast block motion estimation,” Circuits and Systems for Video Technology IEEE Transactions, page(s): 313-317, Jun 1996

[12] Foresti, G. L., Micheloni, C., Snidaro, L., and Marchiol, C., “Face detection for visual surveillance,” in Proc. of the 12th IEEE International Conference on Image Analysis and Processing, Sept. 2003, pp. 115-120.

[13] Chien, S. Y.,Ma S. Y., and Chen, L. G., “Efficient Moving Object Segmenttation Algorithm Using Background Registration Technique,” IEEE Trans. On Circuits and System for Video Technology, Vol. 12, No. 7, July 2002.

[14] Comaniciu, D. and Meer, P., “Mean Shift: A Robust Approach Toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 24, No. 5, May 2002.

[15] Cheng, Y., “Mean Shift, Mode Seeking, and Clustering,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 17(8), pp.790-799, August 1995.

[16] Comaniciu, D., Ramesh, V., and Meer, P., “Real-time tracking of non-rigid objects using mean shift,” IEEE Conference on Computer Vision and Pattern Recognition, vol. II, Hilton Head, SC, June 2000, pp. 142-149.

[17] Fukanaga, K. and Hostetler, L.D., “The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition,” IEEE Trans. Information Theory, Vol 21, pp.32-40, 1975.

[18] Bradski, G.R., “Computer Vision Face Tracking for Use in a Perceptual User

Interface,” IEEE Workshop on Applications of Computer Vision, Princeton, NJ, 1998, pp.214-219.

[19] Collins R. T. “Mean-Shift blob tracking through scale space,” In IEEE International Conference on Computer Vision and Pattern Recognition, 2003, p.234~p.240.

[20] Lindeberg, T., “Feature Detection with Automatic Scale Selectio,” International Journal of Computer Vision, Vol 30(2), pp.79-116, November 1998.

[21] Bretzner, L. and Lindeberg, T., “Qualitative Multiscale Feature Hierarchies for Object Tracking,” Journal of Visual Communication and Image Representation, Vol 11(2), pp.115-129, June 2000.

[22] Katja, N., Esther, K. M., and Luc, V. G., “An Adaptive Color-based Particle Filter, ” Image and Vision Computing, vol. 21, pp.99-110, 2002

[23] Isard, M. and Blake, A., “Contour Tracking by Stochastic Propagation of Conditional Density,” European Conference on Computer Vision, 1996, p.343-356.

[24] Isard, M. and Blake, A., “CONDENSATION Conditional Density Propagation for Visual Tracking,” International Journal on Computer Vision, Page 5-28, 1998 [25] Thacker, N. A., Aherne, F. J. and Rockett, P. I., “The Bhattacharyya Metric as an

Absolute Similarity Measure for Frequency Coded Data,” STIPR97, 1st International Workshop on Statistical Techniques in Pattern Recognition, Prague, Czech Republic, 1997.

[26] Kailath, T., “The Divergence and Bhattacharyya Distance Measures in Signal Selection,” IEEE Transactions on Communication Technology, p.52-60, 1967.

[27] Smith, A. R., ”Color Gamut Transform Pairs,” Computer Graphics, Vol. 12(3),

[28] http://student.kuleuven.be/~m0216922/CG/color.html#The_RGB_Color_Model [29] http://en.wikipedia.org/wiki/Image:HSV_cone.png

[30] http://big5.gov.cn/gate/big5/www.gov.cn/fwxx/ly/2006-01/04/content_146538.htm