Discussion and Comparison with the Homography Technique

In practice, a commonly used technique for camera pose estimation is to find the homography matrix between a reference plane in the 3-D space and the camera’s image plane. The rotation and translation matrices can then be extracted by applying the SVD method over the homography matrix. In the homography approach, we need to define a reference world coordinate system and need to pick up a few spatial points with known reference world coordinates in advance. In other words, not only the distances but also the relative spatial information among the calibration points needs to be known. In comparison, our approach does not need to know the world coordinates of these calibration objects. We only need to measure the lengths or

angles of the calibration objects. Hence, the preparation of calibration objects becomes much easier in our approach.

Besides, we may use fewer spatial points for the calibration of camera pose. This is because there is an implicit constraint in our approach. In a typical setup of PTZ cameras, the horizontal axis of the camera’s image coordinate system is usually parallel to the ground plane. This parallelism is kept all the time even though the camera is under the panning, tilting, and zooming operations from time to time.

Moreover, in our approach, we do not actually care about the exact pan angle of the camera. These two constraints correspond to the constraints over the rotation about the Z axis and the rotation about the Y axis in our rectified world coordinate system.

Hence, our method may lead to stable pose estimations even when we only use a few calibration points.

Fig. 3.16 Test images with a rectangular calibration pattern.

To compare with the homography technique, we marked 20 points on the ground floor to form a rectangular pattern, as shown in Fig. 3.16. Five of these 20 points are chosen to be the calibration points, as marked by the circles in Fig. 3.17. The asterisk markers in Fig. 3.17(a) show the point correspondence based on the calibration result

of the homography technique using the OpenCV library. On the other hand, the asterisk markers in Fig. 3.17(b) show the point correspondence based on our approach using five segments with known lengths. Besides, we calibrated these two cameras twenty times by randomly choosing five of these 20 points as the calibration points.

We then checked the point correspondence in the image captured by Camera 4 based on these 20 image points of Camera 2 and the calibration results. Table 3.4 shows the mean absolute distance and standard deviation of the point-wise correspondence. It can be easily seen that our approach offers a more reliable and stable calibration result even when we only use a few calibration points. When all twenty points are used, on the other hand, there would be no obvious difference between the performance of the homography technique and the performance of our approach. Nevertheless, for general surveillance environments, it will be difficult to place this kind of specific patterns for calibration. Hence, in general, simple calibration objects or a small amount of calibration points without known reference spatial coordinates will be preferred.

(a)

(b)

Fig. 3.17 Evaluation of calibration results by using five points. (a) Point correspondence based on the homography technique. (b) Point correspondence based on the proposed method.

Table 3.4 Mean Absolute Distance and Standard Deviation of the Point-wise Correspondence.

Another advantage of our method is the comprehensible sense of camera pose.

The tilt angle, altitude, and orientation of the camera offer more direct physical sense about the camera pose in the 3D space, especially when the PTZ cameras are under panning or tilting operations from time to time. In our approach, we derive some explicit formula to describe how the tilt angle and altitude of a PTZ camera affects the 3D-to-2D projection. This makes the calculation of 2D-to-3D back-projection much easier without the need of indirect depth computation. Besides, based on the comprehensible space sense, the relationships among multiple cameras can be easily obtained without complicated computations. In comparison, if using the conventional homography technique, the relative position and orientation between each pair of cameras offer less comprehensible sense about the setting of multiple cameras.

Although these relative coordinate systems may still be transformed into an integrated coordinate system, the work for the calibration of multiple cameras will become more and more elaborate when the number of cameras increases.

CHAPTER 4

Dynamic Calibration of Multiple Cameras

______________________________________________

In this chapter, based on the results of our static calibration, a new algorithm for dynamic calibration of multiple cameras is proposed. After the setup of PTZ cameras, we perform static camera calibration first based on the calibration method proposed in Chapter 3. As cameras begin to pan or tilt, we keep extracting and tracking feature points based on the Kanade-Lucas-Tomasi (KLT) algorithm [45]. In Section 4.1, we explain how we utilize the displacement of feature points and the epipolar-plane constraint to infer the changes of pan angle and tilt angle. This algorithm does not require a complicated correspondence of feature points. Our algorithm also allows the presence of moving objects in the captured scenes while performing dynamic calibration. In Section 4.2, we describe how to filter out undesired feature points when moving objects are present. In Fig. 4.1, we show an overall picture of the proposed dynamic calibration algorithm. Besides, the sensitivity analysis with respect to measurement errors and the fluctuations of previous estimations will be addressed in

Section 4.3. Finally, in Section 4.4, the efficiency and feasibility of this approach has been demonstrated in some experiments over real scenery.

Fig. 4.1 Flowchart of the proposed dynamic calibration algorithm.