[4] C. R. Deepak, M. Pugazhaandi, S. Paul, M. A. Shajahan, G. Janaki-raman, M. A. Atmanand, K. Annamalai, R. Jeyamani, M. Ravindran, E. Schulte, J. Panthel, H. Grebe, and W. Schwarz, “Underwater sand mining system for shallow waters,” presented at the 3rd Ocean Mining Symp., Goa, India, Nov. 8–10, 1999.
[5] Z. Janosi and B. Hanamoto, “The analytical determination of drawbar pull as a function of slip for tracked vehicles in deformable soils,” pre-sented at the 1st Int. Conf. Soil Mechanics of Soil-Vehicle Systems, Torino, Italy, 1961.
[6] M. A. Atmanand, M. A. Shajahan, C. R. Deepak, R. Jeyamani, M. Ravindran, E. Schulte, J. Panthel, H. Grebe, and W. Schwarz, “In-strumentation for under water crawler for mining in shallow water,” in Proc. Int. Symp. Autonomous Robots and Agents, Singapore, May 23–26, 2000.
[7] T. Qui and S. C. Jin, “Successive learning track-keeping control (SLTC) algorithm for seafloor vehicle,” in Proc. 9th Int. Offshore and Polar Engineering Conf., Brest, France, Jun. 4, 1999, pp. 441–446. [8] Jong-su Choi, S. Hong, Hyung-Woo Kim, and H. L. Tae, “An
experi-mental study on tractive performance of tracked vehicle on cohesive soft soil,” in Proc. 5th Ocean Mining Symp., Tsukuba, Japan, Sep. 15–19, 2003, pp. 139–143.
[9] G. W. Turnage and W. C. Seabergh, “Traficability and stability analysis for bottom crawling work vehicles in the nearshore region,” in Proc. Offshore Technology Conf., Houston, TX, pp. 1913–1926.
[10] H. Grebe and J. Panthel, “In-situ soil penetrometer,” in Berichte Aus Dem Zentrum für Meeres- und Klimaforschung (Reihe E: Hy-drobiologie und Fischereiwesen, Nr. 11) (in German), G. Schriever, A. Koschinsky, and H. Bluhm, Eds. Hamburg: Inst. Hydrobiolog. Fischereiwesen, 1996, pp. S. 47–55.
[11] O. Bode, Simulation von raupenfahrwerken tiefsee-böden (in German). Hannover, Germany, 1991.
[12] G. Dörfler, Untersuchungen der fahrwerk-boden-interaktion zur gestal-tung von raupenfahrzeugen für die befahrung weicher tiefseeböden (in German). Karlsruhe, Germany, 1995.
[13] J. Y. Wong, Theory of Ground Vehicles, 3rd ed. Singapore: Wiley (Asia), 2001, pp. 560–560.
[14] I Rehorn, “Development of a deep-sea full-track vehicle (Caterpillar vehicle) and testing its internal tractional resistance,” Ph.D. disserta-tion, IKS, Univ. Siegen, Siegen, Germany, 1994.
[15] K. Herzog, Entwicklung Einer Antriebs-Schlupf-Regelung Für ein Tiefseefahrzeug, (in German) Univ. Siegen, Siegen, Germany, 2000.
An Efficient Approach for the Calibration of Multiple PTZ Cameras
I-Hsien Chen and Sheng-Jyh Wang
Abstract—In this paper, we propose an efficient approach to infer the relative positioning and orientation among multiple pan–tilt–zoom (PTZ) cameras. In this approach, the tilt angle and altitude of each PTZ camera are estimated first based on the observation of some simple objects lying on a horizontal plane. With the estimated tilt angles and altitudes, the cal-ibration of multiple cameras can be easily accomplished by comparing the back-projected world coordinates of some common vectors in 3-D space. This calibration method does not require particular system setup or spe-cific calibration patterns. The sensitivity analysis with respect to parameter fluctuations and measurement errors is also discussed. Experiment results over real images have demonstrated the efficiency and feasibility of this approach.
Note to Practitioners—This paper was motivated by the problem that pan–tilt–zoom (PTZ) cameras may change their poses from time to time to achieve better monitoring results. Whenever there is a change, we need to recalibrate the extrinsic parameters again. In this paper, we demonstrate a new and efficient approach to calibrate multiple PTZ cameras. The concept of our approach originated from the observation that people could usually make a rough estimate about the tilt angle of the camera simply based on some clues revealed in the captured images. Based on our approach, we can simply use some A4 papers on a table to calibrate multiple PTZ cameras. In our approach, there is no need to calculate the commonly used homog-raphy matrix. For real cases, once a set of PTZ cameras is settled, we can simply place a few simple patterns on a horizontal plane. These patterns can be A4 papers, books, boxes, etc.; and the horizontal plane can be a tabletop or the ground plane. The whole procedure does not need specially designed calibration patterns and requires only a light computational load. In the near future, we will work on the extension of the proposed approach so that we will be able to perform dynamic calibration when the PTZ cam-eras are under movement.
Index Terms—Camera calibration, multiple cameras, pan-tilt-zoom (PTZ) camera.
I. INTRODUCTION
For a surveillance system with multiple pan–tilt–zoom (PTZ) cam-eras, we may change the cameras’ pan angle and tilt angle from time to time to achieve better monitoring results. However, whenever the pose of a PTZ camera is changed, we may need to recalibrate its extrinsic parameters again. Hence, how to quickly and efficiently calibrate the extrinsic parameters of multiple PTZ cameras has become an impor-tant issue. So far, various kinds of approaches have been developed to calibrate the static camera’s intrinsic and extrinsic parameters, such as the techniques proposed in [1]–[9]. However, it would be impractical to repeatedly perform these elaborate calibration processes over a PTZ camera when that camera is under panning, tilting, or zooming all of the time.
Manuscript received October 23, 2005; revised March 15, 2006. This paper was recommended for publication by Associate Editor Y. F. Li and Editor M. Wang upon evaluation of the reviewers’ comments. This work was supported by the Ministry of Economic Affairs of Taiwan, R.O.C., under Grant No. 95-EC-17-A-02-S1-032.
The authors are with the Department of Electronics Engineering, the Institute of Electronics, National Chiao Tung University, Hsinchu 300, Taiwan, R.O.C. (e-mail: [email protected]; [email protected]).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TASE.2006.884040
On the other hand, [10]–[12] have proposed plane-based calibra-tion methods especially designed for the calibracalibra-tion of multiple cam-eras. These approaches, proposed in [10] and [11], belong to factor-ization-based methods. In [12], the proposed process is similar to the integration of static camera calibration and moving camera calibration. It needs a multicamera rig to change the specific orientation to capture two or more views of a calibration grid. Another kind of method is pro-posed in [13] that uses the relationship between epipoles and trifocal tensors to calibrate multiple cameras. However, their method needs the assumption that at least two cameras are projected to each camera. For surveillance systems with multiple PTZ cameras, these elaborate pro-cesses and the constraints do not seem to be practical choices.
In this paper, we will first show the 3-D-to-2-D coordinate transfor-mation in terms of the tilt angle of a PTZ camera. In [14], a similar scene model based on pan angle and tilt angle has also been established. In this paper, however, we will give a more complete formula that takes into account not only the translation effect but also the rotation effect when a PTZ camera is under tilt movement. We will then infer the tilt angle and the altitude of a PTZ camera based on the observation of some simple objects lying on a horizontal plane. The objects could be a corner with a known angle, a few corners with unknown but equal angles, a line segment with a known length, or a few line segments with unknown but equal lengths. In comparison, [15] proposed an approach that recovers the transformation between the image and the ground plane to find the look-down angle. In their method, however, the height information of the observed object has to be known in advance. In comparison, the height is not a necessity in our approach. After having calibrated the tilt angle and the altitude of each PTZ camera, we can easily achieve the calibration of multiple PTZ cameras based on some commonly ob-served objects. In some sense, our approach can be thought of to have decomposed the computation of homography matrix into two simple calibration processes so that the computational load becomes lighter for the calibration of multiple PTZ cameras.
This paper is organized as follows. First, in Section II, we will de-scribe the mapping between the horizontal plane in the 3-D space and the 2-D image plane on a tilted camera. Based on the back projection formula, the pose (the tilt angle and the altitude) of the camera can be estimated by viewing some simple patterns on a horizontal plane. Next, in Section III, we will introduce how to utilize the estimation results to achieve the calibration of multiple PTZ cameras. The sensitivity anal-ysis with respect to parameter fluctuations and measurement errors will be discussed in Section IV. Some experimental results on real data are demonstrated in Section V to illustrate the feasibility of this method. Finally, the conclusion is drawn.
II. POSEESTIMATION OF APTZ CAMERA
In this section, we show the mapping between the 3-D space and the image plane on a tilted PTZ camera, under the constraint that all observed points are located on a horizontal plane. We will then demon-strate that, with a few corners or a few line segments lying on a hori-zontal plane, we can easily estimate the tilt angle of the camera based on the back projection of the captured image.
A. Constrained Coordinate Mapping on a Tilted Camera
Fig. 1 illustrates the modeling of our camera setup. Here, we assume that the observed objects are located on a horizontal plane , while the PTZ camera lies above with a heighth. The PTZ camera may pan or tilt with respect to the rotation centerOR. Moreover, we assume the projection center of the camera, denoted asOC, is away fromORwith distancer. We define the origin of the rectified world coordinates to be the projection centerOCof a PTZ camera with zero tilt angle. The Z-axis of the world coordinates is along the optical axis of the camera, while theX- and Y -axis of the world coordinates are parallel to the
x-Fig. 1. Model of camera setup.
andy-axis of the projected image plane, respectively. When the camera tilts, the projection center moves toOC0 and the projected image plane is changed to a new 2-D plane.
AssumeP = [X; Y; Z; 1]T denotes the homogeneous coordinates of a 3-D pointp in the world coordinates. For the case of a PTZ camera with zero tilt angle, we denote the perspective projection of p as p = [x; y; 1]T. Under perspective projection, the relationship betweenP
andp can be expressed as p = 1Z
s u0
0 v0
0 0 1
[ R t ]P (1)
where and are the scale parameters expressed in pixel units for thex and y axes in the image plane, s is the skew factor due to some manufacturing error, and(u0; v0) is the principal point [16].
With respect to the rectified world coordinate system, the extrinsic term[ R t ] becomes [ I 0 ]. To further simplify the mathematical deduction, we ignore the skew factors and assume the image coordi-nates have been translated by a translation vector(0u0; 0v0). Hence,
(1) can be simplified and reversed as X Y Z = Z 1 0 0 0 1 0 0 0 1 x y 1 = xZ yZ Z : (2)
When the PTZ camera tilts with an angle , the homogeneous co-ordinates of the projected image point will move to[x0; y0; 1]T. In our previous work [17], we obtained (3). This equation indicates the back projection formula from the image coordinates of a tilted camera to the rectified world coordinates, under the constraint that all of the observed points are lying on a horizontal plane withY = 0h
X Y Z = (x 0u ) (r sin 0h) [(v 0y ) cos 0 sin )] 0h
[(v 0y ) sin + cos ](r sin 0h)
(v 0y ) cos 0 sin 0 r + r cos
: (3)
B. Estimation of Pose Based on Back Projections
Suppose we use a tilted camera to capture the image of a corner, which is located on a horizontal plane. Based on the captured image and a guessed tilt angle, we may use (3) to back project the captured image onto a horizontal plane onY = 0h. Assume three 3-D points PA,PB, andPC, on a horizontal plane form a rectangular corner atPA. The original image is captured by a camera with = 16, as shown in Fig. 2(a). In Fig. 2(b), we plot the back-projected images for various
Fig. 2. (a) Rectangular corner captured by a tilted camera. (b) Illustration of back projection onto a horizontal plane for different choices of tilt angles.
choices of tilt angles. The guessed tilt angles range from 0 to 30, with a 2step. The back projection for the choice of 16is plotted in bold, specifically. It can be seen that the back-projected corner becomes a rectangular corner only if the guessed tilt angle is correct. Besides, it is worth mentioning that a different choice ofh only causes a scaling effect of the back-projected shape.
To formulate this example, we assume the corner atPAhas the angle . Besides, PA andPB form a line segment with lengthL. In [17], we have built the relation between the back-projected angle and the guessed tilt angle as shown in (4) at the bottom of the next page. Note that in (4) we have ignored the offset terms,u0andv0, to reduce the complexity of the formulation.
In Fig. 3, we show the back-projected angle with respect to the guessed tilt angle, assuming and are known in advance. In this simulation, these two curves are generated by placing the rectangular corner on two different places of the horizontal plane. Again, the back-projected angle is equal to 90only if we choose the tilt angle to be 16. This simulation demonstrates that if we know in advance the angle of the captured corner, we can easily deduce the camera’s tilt angle. Moreover, if we do not know in advance the actual angle of the corner, we can simply place that corner on more than two different places of the horizontal plane. In this way, based on the intersection of the deduced -v.s.- curves, we may estimate not only the tilt angle of the camera but also the actual angle of the corner.
In [17], we have also derived a similar relationship between the back-projected length and the guessed tilt angle as
L = l( ) = (x0B0 u0) (r sin 0 h)
[(v00 yB0 ) cos 0 sin ]
0 (x0A0 u0) (r sin 0 h)
[(v00 y0A) cos 0 sin ] 2
+ [(v00 yB0 ) sin + cos ](r sin 0 h)
(v00 yB0 ) cos 0 sin
0 [(v00 y0A) sin + cos ](r sin 0 h)
(v00 y0A) cos 0 sin
2 1=2
: (5)
Fig. 3. Back-projected angle with respect to guessed tilt angles.
Similarly, if , , r, and h are known in advance, we can deduce the tilt angle directly based on the projected value ofL. Note that in (5), the values ofr and h only affect the scaling of L. Hence, even if the values ofr and h are unknown, we may simply place more than two line segments of the same length on different places of a horizontal plane and seek to find the intersection of these correspondingL-v.s.- curves.
We may also seek to perform parameter estimation based on an opti-mization process. Here, we take (5) as an example. We assume several line segments with known lengths (not necessarily the same length) are placed on different positions of a horizontal plane and we use a tilted camera to capture the image. Assume the length of theith segment is Li, then we aim to find a set of parametersf ; ; u0; v0; ; r; hg that
minimize F (x0 1; y01; x02; y20; . . . ; x0m; y0m; ; ; u0; v0; ; r; h) = m i li x0i; y01; ; ; u0; v0; ; r; h 0 Li 2: (6)
In the optimization process, we adopt the Levenberg–Marquardt algo-rithm. Under our scene model, the tilt angle and altitude h can be roughly estimated simply based on visual observations. In our experi-ments, the error range for the guessed tilt angle is within 620and the error range for the guessed altitudeh is within 61.5 m. With these initial guesses, the optimization process is very stable and the estima-tion results are satisfactorily accurate.
III. CALIBRATION OFMULTIPLECAMERAS A. Calibration of Multiple PTZ Cameras
In our camera model, each camera has its own world coordinate system. If a vector in the 3-D space, such as a line segment on a tabletop, is observed by several PTZ cameras at the same time, we can achieve the calibration of these cameras by mapping the individual back-pro-jected world coordinates of this vector to a common reference world co-ordinates. We take two calibrated PTZ cameras as an example. Fig. 4(a) shows the scene model of these two cameras. Fig. 4(b) shows the vector locations in the world coordinates of these two cameras, respectively. Based on the estimated and h, together with the image projections of the vector points, we can get the world coordinates of pointsAref, Bref, andA0,B0from (3). The difference of the rotation angle!
Fig. 4. (a) Top view of two cameras and a vector in the space. (b) The world coordinates of the vector with respect to these two cameras.
by
cos w = kAhA0B00Bk 2 kA0; ArefBrefi
refBrefk: (7)
After applying the rotation to pointA0, the position translationt be-tween these two cameras can be expressed as
t = Aref0
cos w 0 0 sin w
0 1 0
sin w 0 cos w
1 A0: (8)
Hence, the 3-D relationship between these two cameras can be easily deduced.
B. Discussion of Pan Angle
Notice that, in the above deductions, we do not care about the pan angles of PTZ cameras. This is because, in the back-projection process, to guess a pan angle only implies to rotate theX and Z coordinates in
our camera model. It does not change the spatial relationship between the PTZ camera and the back-projected objects. In Fig. 5, we show such an example. In Fig. 5(a), we show the image captured by a PTZ camera with three corner points being marked in circle. Fig. 5(b) shows the top view (i.e.,X-Z plane) of the back-projected corner points with respect to four guessed pan angles, 0, 30, 60, and 90. The arrows indicate the optical axes of the camera with respect to these four pan angles. It can be seen that the spatial relationship between the optical axis and the back-projected corner points is almost the same when the camera pans. The little variation comes from the fact that the panning center is not the same as the projection center. However, since the rotation radiusr is so small when compared with the distance between the camera and the object, this small variation can actually be ignored.
IV. SENSITIVITYANALYSIS
Due to errors in the estimation of camera parameters and in measure-ment, these -v.s.- curves or L-v.s.- curves usually do not intersect at a single point. The impacts of parameter fluctuations have been an-alyzed via computer simulations in [18]. Here, we will deduce the for-mulae for the sensitivity analysis of the estimated and h.
In theory, for the estimation of and h, the optimization of (6) con-forms to the following two equations:
f1= m i=1 @l @ 0 m i=1Li @l @ = 0 f2= m i=1 @l @h 0 m i=1Li @l @h = 0: (9)
Hence, the estimated and h satisfy f1( ; h) = 0 and f2( ; h) = 0. We apply the implicit function theorem to (9) to find how and h deviate with respect to the measurement error inx0i
@ @x @h @x = 0 @f@ @f@h @f @ @f@h 01 @f @x @f @x : (10) = cos01 x0B (y0 Bcos 0 sin )0 x0 A (y0 Acos 0 sin ) 2 x0C (y0 Ccos 0 sin )0 x0 A (y0 Acos 0 sin ) + (y0Bsin + cos ) y0 Bcos 0 sin 0 (y 0 Asin + cos ) y0 Acos 0 sin 2 (y0Csin + cos ) y0 Ccos 0 sin 0 (y 0 Asin + cos ) y0 Acos 0 sin 2 x0B (y0 Bcos 0 sin )0 x0 A (y0 Acos 0 sin ) 2 + (y0Bsin + cos ) y0 Bcos 0 sin 0 (y 0 Asin + cos ) y0 Acos 0 sin 2 01=2 2 x0C (y0 Ccos 0 sin )0 x0 A (y0 Acos 0 sin ) 2 + (y0Csin + cos ) y0 Ccos 0 sin 0 (y 0 Asin + cos ) y0 Acos 0 sin 2 01=2 : (4)
Fig. 5. (a) Three points marked in the image captured by a PTZ camera. (b) Top view of the back-projected corners and the optical axes with respect to different guessed pan angles.
Similarly, we can deduce the formulae for@ =@ , @ =@ , and so on. If we assume the total variations of and h are caused by individual variations with respect to parameter fluctuations inf ; ; u0; v0g and
measurement errors infx0i; yi0g, then we have 1 m i=1 @ @x0 i1x 0 i + m i=1 @ @y0 i1y 0 i + @ @ 1 + @ @ 1 + @ @u 01u0+ @ @v01v0 (11) and 1h m i=1 @h @x0 i1x 0 i + m i=1 @h @y0 i1y 0 i + @h@ 1 + @h@ 1 + @h@u 01u0+ @h@v01v0: (12)
To verify the formulae deduced above, we perform the following simulations. Here, we assume two line segments of 0.283 m are placed on a horizontal plane and the altitudeh = 2 m. The values offu0; v0; ; g are {348, 257, 770, 750}. The images of these two
TABLE I
VARIATIONS OFTILTANGLE ANDALTITUDEWITHRESPECT TODIFFERENT
PARAMETERFLUCTUATIONS ANDMEASUREMENTERRORS
segments are assumed to be captured by a PTZ camera with the tilt angle = 60.
In the first simulation, the tilt angle = 60. We individually change the values of camera parameters and the measurement of(x0i; yi0) to see how the estimated values of and h vary. Here, the LM algorithm is applied to (6) for the estimation of and h. The variations of these estimation results, together with the variations deduced by (11) and (12) are listed in Table I. It can be seen that the deduced variations based on (11) and (12) well approximate the simulation results.
In the second simulation, we change the tilt angle from 15 to 75, with a 15step. Table II demonstrates that (11) and (12) conform to the variations of the simulation results for a wide rage of tilt angle . In practice, the fluctuations of camera parameters are likely to be less than 20 and the measurement errors of(x0i; y0i) are likely to be less than 4 pixels. Hence, the estimation errors of and h are expected to be acceptable in real cases.
V. EXPERIMENTS ONREALIMAGES
The parameter estimation results of a single PTZ camera have been shown and discussed in [18]. In this section, some calibration results of multiple cameras on real data are demonstrated. In this simulation, the test images are captured by a PTZ camera mounted on the ceiling with an unknown tilt angle. The image resolution is 320 by 240 pixels. A few A4 papers are randomly placed on a horizontal table, as shown in Fig. 6. The corners of these A4 paper sheets could be easily identified either by hand or by a corner detection algorithm. So far, we identify these corners manually and we have developed a software package to facilitate the identification of corners and line segments in images. In the 3-D space, all of the corners are 90, while the length and width of an A4 paper are 297 mm and 210 mm, respectively.
The upper part of Table III lists some experimental results for the calibration of multiple cameras. Here, the intrinsic parameters f ; ; u0; v0g of each camera are estimated in advance, based on
Zhang’s calibration method [3]. The parameterr can be estimated via direct measurement. Hence, (6) includes only two unknown variables and h. Each row of Table III lists the mean and standard deviation of the estimated parameters for a single camera. To calculate the
TABLE II
VARIATIONS OFTILTANGLE ANDALTITUDE WITH
RESPECT TODIFFERENTCHOICES OFTILTANGLE
mean and standard deviation, five observations are made with each observation including eight selected line segments on the boundary of these A4 papers, as shown in Fig. 6(a). It can be seen that all of the estimated parameters have an acceptably small standard deviation.
In the lower part of Table III, each row corresponds to the estimations of the position and orientation of each camera with respect to Camera 2. The relative position and orientation are computed based on the mean value of and h and one common vector in Fig. 6(a). The top view of the relative positions in the 3-D space is illustrated in Fig. 6(b). The eight chosen corners of the A4 papers in Fig. 6(a) are also plotted in Fig. 6(b) to offer a clearer geometric sense.
To evaluate the calibration results, we randomly pick up a few test points in the image captured by Camera 2 and use the calibration result to find the corresponding points on the other three images. The results are shown in Fig. 7, with all corresponding points being represented by the same type of marker. It can be seen that all of the corresponding relationships are reasonably accurate.
In practice, a commonly used technique for camera pose estimation is to find the homography matrix between a reference plane in the 3-D space and the camera’s image plane [19]. The rotation and translation matrices can then be extracted by applying the singular value decompo-sition (SVD) method over the homography matrix. In the homography approach, we need to define a reference world coordinate system and need to pick up a few spatial points with known reference world coor-dinates in advance. In other words, not only the distances but also the relative spatial information among the calibration points needs to be known. In comparison, our approach does not need to know the world coordinates of these calibration objects. We only need to measure the lengths or angles of the calibration objects. Hence, the preparation of calibration objects becomes much easier in our approach.
Besides, we may use fewer spatial points for the calibration of camera poses. This is because there is an implicit constraint in our approach. In a typical setup of PTZ cameras, the horizontal axis of the camera’s image coordinate system is usually parallel to the ground plane. This parallelism is kept all of the time even though the camera is under the panning, tilting, and zooming operations from time to time. Moreover, in our approach, we do not actually care about the exact pan angle of the camera. These two conditions correspond to the constraints over the rotation about the Z axis and the rotation about theY axis in our rectified world coordinate system. Hence, our
Fig. 6. (a) Test image captured by four PTZ cameras. (b) Top view of the rel-ative positions between four PTZ cameras.
TABLE III
UPPER: ESTIMATEDTILTANGLE ANDALTITUDE. LOWER: SPATIALRELATIONSHIPBETWEENCAMERAS
method may lead to stable pose estimations even when we only use a few calibration points.
Fig. 7. Evaluation of calibration results.
Fig. 8. Test images with a rectangular calibration pattern.
Fig. 9. Evaluation of calibration results by using five points. (a) Point corre-spondence based on the homography technique. (b) Point correcorre-spondence based on the proposed method.
To compare with the homography technique, we marked 20 points on the ground floor to form a rectangular pattern, as shown in Fig. 8. Five of these 20 points are chosen to be the calibration points, as marked by the circles in Fig. 9. The asterisk markers in Fig. 9(a) show the point correspondence based on the calibration result of the homog-raphy technique using the OpenCV library. On the other hand, the as-terisk markers in Fig. 9(b) show the point correspondence based on our approach using five segments with known lengths, as shown in Fig. 9(b). Besides, we calibrated these two cameras 20 times by ran-domly choosing 5 of these 20 points as the calibration points. We then
TABLE IV
MEANABSOLUTEDISTANCE ANDSTANDARDDEVIATION OF THE
POINT-WISECORRESPONDENCEBETWEENCAMERA2ANDCAMERA4
checked the point correspondence in the image captured by Camera 4 based on these 20 image points of Camera 2 and the calibration re-sults. Table IV shows the mean absolute distance and standard devi-ation of the point-wise correspondence. It can be easily seen that our approach offers a more reliable and stable calibration result when we only use a few calibration points. When all 20 points are used, on the other hand, there would be no obvious difference between the perfor-mance of the homography technique and the perforperfor-mance of our ap-proach. Nevertheless, for general surveillance environments, it could be difficult to place this kind of specific pattern for calibration. Hence, in general, simple calibration objects or a small amount of calibration points without known reference spatial coordinates will be preferred.
Another advantage of our method is the comprehensible sense of camera pose. The tilt angle, altitude, and orientation of the camera offer a more direct physical sense about the camera pose in the 3-D space, especially when the PTZ cameras are under panning, tilting, and zooming operations from time to time. In our approach, we de-rive some explicit formula to describe how the tilt angle and altitude of a PTZ camera affect the 3-D-to-2-D projection. This makes the cal-culation of 2-D-to-3-D backprojection much easier without the need of indirect depth computation. Besides, based on the comprehensible space sense, the relationships among multiple cameras can be easily ob-tained without complicated computations. In comparison, when using the conventional homography technique, the relative position and ori-entation between each pair of cameras offer less comprehensible sense about the setting of multiple cameras. Although these relative coor-dinate systems may still be transformed into an integrated coorcoor-dinate system, the work on the calibration of multiple cameras will become more elaborate when the number of cameras increases.
VI. CONCLUSION
To summarize, in this paper, we adopt a system model that is gen-eral enough to fit for a large class of surveillance systems with multiple PTZ cameras. A calibration method is proposed that does not require a particular system setup or specific calibration patterns. When com-pared to some existing calibration approaches, which extract the ho-mography matrix and the rotation matrix, our approach offers direct geometric sense and can simplify the calibration process. No coordi-nated calibration pattern is needed and the computational load is light. The experiment results over real images demonstrate the efficiency of this approach.
REFERENCES
[1] R. Y. Tsai, “A versatile camera calibration technique for high-accu-racy 3D machine vision metrology using off-the-shelf TV cameras and lenses,” IEEE J. Robot. Autom., vol. RA-3, no. 4, pp. 323–344, Aug. 1987.
[2] M. Agrawal and L. S. Davis, “Camera calibration using spheres: A semi-definite programming approach,” in Proc. 9th IEEE Int. Conf. Computer Vision, Oct. 2003, vol. 2, pp. 782–789.
[3] Z. Zhang, “Flexible camera calibration by viewing a plane from un-known orientations,” in Proc. 7th IEEE Int. Conf. Computer Vision, Sep. 1999, vol. 1, pp. 666–673.
[4] G.-Q. Wei and S. D. Ma, “A complete two-plane camera calibration method and experimental comparisons,” in Proc. 4th Int. Conf. Com-puter Vision, May 1993, pp. 439–446.
[5] P. F. Sturm and S. J. Maybank, “On plane-based camera calibration: A general algorithm, singularities, application,” presented at the IEEE Comput. Soc. Conf. Computer Vision and Pattern Recognition, Jun. 1999.
[6] Z. Zhang, “Camera calibration with one-dimensional objects,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 7, pp. 892–899, Jul. 2004.
[7] A. Basu and K. Ravi, “Active camera calibration using pan, tilt and roll,” IEEE Trans. Syst., Man Cybern. B, Cybern., vol. 27, no. 3, pp. 559–566, Jun. 1997.
[8] E. E. Hemayed, “A survey of camera self-calibration,” in Proc. IEEE Conf. Advanced Video and Signal Based Surveillance, Jul. 2003, pp. 351–357.
[9] R. T. Collins and Y. Tsin, “Calibration of an outdoor active camera system,” in Proc. IEEE Comput. Soc. Conf. Computer Vision and Pat-tern Recognition, Jun. 1999, vol. 1, pp. 534–534.
[10] P. Sturm, “Algorithms for plane-based pose estimation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Jun. 2000, vol. 1, pp. 706–711.
[11] T. Ueshiba and F. Tomita, “Plane-based calibration algorithm for multi-camera systems via factorization of homography matrices,” in Proc. 9th IEEE Int. Conf. Computer Vision, Oct. 2003, vol. 2, pp. 966–973.
[12] C. Wiles and A. Davison, “Calibrating a multi-camera system for 3D modelling,” in Proc. IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, Jun. 1999, pp. 29–36.
[13] Y. Sugimure and J. Sato, “Camera calibration and reconstruction from the chain connection of mutual camera projections,” in Proc. 17th Int. Conf. Pattern Recognition, Aug. 2004, vol. 1, pp. 100–103.
[14] S.-N. Lim, L. S. Davis, and A. Elgammal, “A scalable image-based multi-camera visual surveillance system,” in Proc. IEEE Conf. Ad-vanced Video and Signal Based Surveillance, Jul. 2003, pp. 205–212. [15] P. Remagnino and G. A. Jones, “Automated registration of surveillance
data for multi-camera fusion,” in Proc. 5th Int. Conf. Information Fu-sion, Jul. 2002, vol. 2, pp. 1190–1197.
[16] D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Upper Saddle River, NJ: Prentice-Hall, 2003, pp. 28–30.
[17] I.-H. Chen and S.-J. Wang, “Efficient vision-based calibration for vi-sual surveillance systems with multiple PTZ cameras,” in Proc. IEEE Int. Conf. Computer Vision Systems, Jan. 2006, pp. 24–31.
[18] I.-H. Chen and S.-J. Wang, “A vision-based approach to extracting the tilt angle and altitude of a PTZ camera,” Proc. SPIE, vol. 6066, pp. 606609-1–9, Jan. 2006.
[19] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge University Press, 2003, ch. 7.
