Camera Calibration with a Motorized Zoom Lens

(1)

Camera Calibration with a Motorized Zoom Lens

Yong-Sheng Chentt Sheng-Wen Shih* Yi-Ping Hungtt Chiou-Shann Fuht

tInstitute of Information Science, Academia Sinica, Taipei, Taiwan

$Dept. of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan

*Dept. of Computer Science and Information Engineering, National Chi Nan University, Nantou, Taiwan Email: [email protected]

Abstract

This paper presents a simple and eficient method of cal- ibrating the intrinsic camera parameters for all the lens settings of a motorized zoom lens. We

fu

the aperture set- ting and perform the camera calibration, adaptively, over the ranges of the zoom and focus settings. Bilinear interpo- lation is used to provide the values of the intrinsic camera parameters for those lens settings where no observations are taken. Our experiments show that the proposed method can provide accurate intrinsic camera parameters for all the lens settings, even though camera calibration is performed only for a small number of sampled lens settings. A calibration object suitable for zoom lens calibration is also presented.

1. Introduction

Motorized zoom lenses have great potential in the applications of active vision [3], 3-D reconstruction [2], and tracking [ 11. In such applications, the aperture, zoom, and focus of the lens can be controlled to adapt to different lighting conditions or to obtain the desired field of view, depth of field, spatial resolution, or focused distance. Although a motorized zoom lens is more flexible and useful than a monofocal lens, it is not an easy job, in general, to Cali- brate a motorized zoom lens. The goal of motorized zoom lens calibration is to determine the relationship between the lens settings (control parameters for the driving motors) and the intrinsic camera parameters (ICPs). Unfortunately, a motorized zoom lens usually consists of some compound lens groups and various mechanical assembly. The relationship between the lens settings and the ICPs is quite com- plicated [6]. One way to determine this relationship is to treat each configuration of lens settings as a monofocal lens and to perform camera calibration [5] for each configura- tion. However, this method is extremely inefficient because a motorized zoom lens usually has many configurations.

In the past, Tarabanis et al. [4] proposed techniques for zoom lens calibration by using a special optical bench.

They constructed a sparse table storing the camera parameters (CPs) calibrated for sampled lens settings. CPs for other lens settings can be obtained via interpolation. Willson and Shafer [6] also have developed zoom lens calibration tech- nique. They used an autocollimated laser for locating the image center and calibrated the eleven CPs for regularly sampled lens settings. For each CP, they approximately modeled the relationship between the CP and the lens settings (zoom and focus) with a bivariate polynomial function.

The calibrated CPs of the sampled lens settings were then used to determine the coefficients of the polynomial functions. These polynomial functions can provide accurate CPs for a continuous range of lens setting. Average prediction error of less than 0.14 pixels was achieved.

In this paper, we present a simple and efficient method of calibrating a motorized zoom lens. A small number of sampled lens settings are adaptively chosen for performing camera calibration. The calibrated ICPs are stored in a table.

For those lens settings where the table entries are empty, the desired ICPs can be obtained via interpolation. This table can be used to provide accurate ICPs of any lens setting for many applications.

2. Camera Model

Given a 3-D point in the world coordinate system, the camera model and the associated CPs can predict its projected image coordinates. For a monofocal lens, here, the pinhole camera model with radial distortion is considered.

Totally, there are twelve CPs in this camera model: the image coordinates, ( U O , V O ) , of the piercing point where the optical axis pierces the image plane; the horizontal and vertical pixel width, s, and s,; the effective focal length, f ; the first coefficient of radial distortion, ^{6 ;}the X-Y-Z Euler angles, a ,

p,

and y; and the translation vector between the origins of the world and the camera coordinate systems, t,,

0-7695-0750-6/00 $10.00 0 2000 IEEE 495

Authorized licensed use limited to: National Taiwan University. Downloaded on December 19, 2009 at 04:42 from IEEE Xplore. Restrictions apply.

(2)

t,, and t,. Among these CPs, the vertical pixel width, sur can be obtained from the specification of the CCD camera.

The remaining eleven CPs can be estimated by using Weng et al.’s method [5], provided that a set of known calibration points are observed in the image. In the following, the 3-D world coordinates of the calibration points and their corresponding image measurements will be referred to as the calibration data.

Since the influence of the aperture setting is negligible, as shown in Section 4, we only consider the zoom and focus settings. For each combination of the zoom and focus settings, the zoom lens can be treated as a monofocal lens and its CPs can be calibrated individually. When we adjust the zoom or the focus setting, some of the CPs remain unchanged during the adjustment. For example, the pixel width, s, and svr and the extrinsic CPs, a,

p,

7, t,, t,, and t,, are not supposed to vary. Hence, once these CPs are determined in the initial stage, they can be fixed during the remaining procedure of zoom lens calibration. When adjusting the zoom or the focus setting, the ICPs, uo, 2ro, j,

IE, and At,, will change accordingly, where At, is the dis- placement of the perspective center in the Z axis. We should estimate these ICPs for each combination of the zoom and the focus settings.

3. Zoom Lens Calibration

The calibration object we used is a plate mounted on a computer-controlled translation stage, as shown in Figure 1.

This plate can be moved along the direction of the stage such that the calibration data of different distances from the lens can be obtained. On the calibration plate, there are many circles used as the calibration pattems. Each circle projected in the image plane will be “symmetrically” blurred if the circle is out of focus. This defocusing problem can be avoid by measuring the centroids of the circles in the image as the image coordinates of the calibration data.

To remedy the problem of varying field of view and spatial resolution during the adjustment of the zoom and focus, circles of two different sizes were used. When the zoom of the lens is set to be wide-angle, many larger circles as well as the smaller circles in the middle of the calibration plate appear in the image, as shown in Figure l(b). These larger circles can provide more accurate centroid positions estimated in the image. When the zoom of the lens, on the other hand, is set to be telephoto, only a few smaller circles in the middle of the calibration plate appear in the image, as shown in Figure l(c). These smaller circles still can provide accurate centroid positions due to larger magnification.

Image coordinates of the calibration data, i.e., the center of the circles, can be estimated by using simple image pro- cessing techniques. Considering the varying field of view, the calibration plate may partially appear in the image. We

Figure 1. (a) The calibration object we used and its images acquired with (b) wide-angle and (c) telephoto zoom settings.

need to identify the circles appearing in the image in order to obtain their corresponding 3-D world coordinates. For easy identification, the larger circle in the middle surrounding by twenty-four smaller circles is chosen to be the fiducial circle. The origin of the world coordinate system is set to the center of the fiducial circle. The 3-D coordinates of the other circles are measured in advance according to this origin. If the lens is properly aligned such that the fiducial circle always appear in the image for all the zoom settings, it will be easy to locate the fiducial circle by utilizing the sizes of the circles. Each circle appearing in the image can then be identified and its 3-D world coordinates can be obtained.

In our work, there are three thousand steps for both the zoom and focus motors and totally nine million lens settings in combination. To reduce the frequency of camera calibration performed and the storage required to store the ICPs, sampling and interpolation over the range of the lens settings should be used. Therefore, our goal is to create a two-dimensional table indexed by the values of the lens settings to store the ICPs estimated at the sampled lens settings. The ICPs of the other lens settings where the actual camera calibration is not performed can be approximated via bilinear interpolation between the four neighboring lens settings in the table.

Since the ICPs vary nonlinearly with respect to the lens setting, an uniformly sampled table might not represent their relationship very well. A better sampling policy is the “calibration on demand” that allows adaptive sampling positions and variable sampling rate. Therefore, the table will be adaptively created by trial and error to store the ICPs estimated at sampled lens settings. Our decision criterion is based on the residual error, which is the difference between the predicted and measured image coordinates of the calibration data.

Figure 2 gives an example of how we determine the sampled positions. At first, we adjust the aperture stop for suitable lighting condition and then fix the aperture stop.

496

(3)

focus setting

FEND 2

1 1 14 6

FREF

FSE4Rl 1 12 4

mom scttmng

" 7 E N D

Figure 3. Our motorized zoom lens.

ibration data are estimated. Then, we calculate the residual

L

Figure 2. An example

of

sampled positions.

Then, we set the zoom setting to be the middle of its range, [ Z S T A R T , Z E N D ] . Next, the calibration plate is moved to the middle of its range and the focus setting is adjusted in its range, [ F S T A R T , F E N D ] , until the image of the calibration plate is sharpest. This focus setting is referred to as F R E F . Camera calibration is then performed (step 0 shown in Figure 2) to obtain all the CPs. During the following zoom lens calibration procedure, the CPs, s%, s,, a , ,L?, y, t,, t,, and t , are fixed and the remaining ICPs, ^{U O ,}^{W O ,}

f,

K , At,, are calibrated.

For each zoom setting to be calibrated, camera calibration is first performed at the start and end of the focus setting, for example, steps 1 and 2 shown in Figure 2. The obtained ICPs and residual errors are stored in the table. Then the focus setting is set to the middle of its range. The ICPs and the residual error, perror, at this focus setting can be interpolated by using the ICPs and residual errors previously obtained at the start and end of the focus setting. Next, the images of the calibration plate are acquired and the 2-D image coordinates of the calibration data are estimated. Then, we compute the residual error, f error, between the estimated image coordinates and the predicted positions calculated from the interpolated ICPs. Large ferror means the interpolated ICPs are not accurate and the camera calibration has to be performed (step 3 shown in Figure 2). This procedure is recursively repeated for each middle focus setting between two calibrated focus settings (for example, steps

4

to 8 in Figure 2) until the interpolated ICPs are accurate, i.e., ferror is small enough.

Because the residual errors of camera calibration are not the same for all lens settings, the decision that f error is small enough or not is adaptively determined by the criterion:

ferror

<

(1

+

F R A T I O )

*

perror.

We can adaptively calibrate for the zoom setting in the same way as for the focus setting. The start and end of the zoom setting are first calibrated by using the previously mentioned procedure, which is illustrated by the steps of the first and the last columns shown in Figure 2. Then, the zoom setting is set to be the middle of its range and the focus setting is set to F R E F . Next, the images of the calibration plate are acquired and the 2-D image coordinates of the cal-

error, zerror, between the estimated image coordinates and the predicted positions calculated by using the interpolated ICPs among the first and last columns in the table. If zerror is small, i.e., zerror

<

(1

+

Z R A T I O ) *perror, the interpolated ICPs is accurate. Otherwise, camera calibration has to be performed to obtain accurate ICPs (the middle column shown in Figure 2). Again, this procedure is repeated recur- sively for each middle zoom setting between two calibrated zoom settings until the interpolated ICPs are accurate, i.e., zerror is small enough.

Two parameters, F R A T I O and Z R A T I O , determine the sampling rate and the sampling positions of the constructed table. They control the maximum difference of the residual errors between the ICPs calculated via actual camera calibration and bilinear interpolation.

4. Experimental Results

In this work, we constructed a motorized zoom lens by using a Fujinon TV zoom lens H6 x 12.5R, as shown in Figure 3. Three servo motors were used for driving the aperture, zoom, and focus of the lens through pushrod links.

We adopted servo motor because of its faster response, de- spite of the disadvantage of poor repeatability due to the large dead zone.

To avoid the hysteresis problem [6] due to the backlash, we set the motor to the desired setting in always the same direction. That is, the motor is first set to the setting a little before the desired one and then set to the desired setting.

The aperture experiment was conducted to explore the relationship between the aperture setting and the CPs. The zoom setting was set to the middle of its range and the focus was adjusted to obtain the sharpest image of the calibration plate. Then, we estimated the CPs at a specific aperture setting (1500). For other six aperture settings, images of the calibration plate were taken and the image coordinates of the calibration data was measured. The image coordinates can be predicted by using the previously estimated CPs and the residual error from the measured image coordinates was found relatively small, as depicted in Figure 4. The average

497

(4)

I to*

. . ._. .

Figure 4. Residual for aperture settings.

. . . . . .

amount of the increased residual error is 0.0125 pixels. We conclude that the influence of the aperture setting on the CPs is negligible and the aperture setting were then fixed during the proposed calibration procedure.

In another experiment, we performed the proposed zoom lens calibration method for our motorized zoom lens. The parameters F R A T I O and Z R A T I O were set to 0.5 and 1, respectively. A table with 120 entries was created and the stored ICPs are depicted in Figure 5, where t z O is the value of the fixed CP t , obtained in the initial camera calibration. The mean residual error in the table is 0.19 pixels. To evaluate the repeatability of the motorized zoom lens, the sampled lens settings in the table were set again and the images of the calibration palate were taken. We then calculated the residual errors between the measured image coordinates in these images and the predicted positions by using the fixed CPs and the ICPs provided by the table. Mean residual error in this repeatability experiment is 0.44 pixels. Also, four hundred trials of residual error evaiuation with random lens settings were performed and the mean residual error obtained is 0.475 pixels, which is only 8% increase of the repeatability error.

Another experiment was conducted with different values of the parameters F R A T I O and Z R A T I O , which were set to 1 and 2 respectively. Since the amount of increasing residual error allowed was larger, a table with smaller size (52 entries) was created at the expense of higher interpola- tion error, which was 0.54 pixels (22.7% increase) in this experiment.

Figure 5. The lCPs stored in the table.

iments, the average residual error of the camera parameters given by the table (with only 120 entries) is less than half pixel. Another contribution of this work is the proposed calibration object suitable for zoom lens calibration.

Acknowledgments

Dr. Shih would like to thank Academia Sinica for the support of his summer visit. This work was also supported in part by the National Science Council of Taiwan, under Grants NSC 88-2213-E-001-010. The authors would like to thank the reviewers for their helpful comments.

References

[l] J. A. Fayman, 0. Sudarsky, and E. Rivlin. Zoom tracking.

In Proceedings of International Conference on Robotics and Automation, pages 2783-2788, Leuven, Belgium, May 1998.

[2] J.-M. Lavest, G. Rives, and M. Dhome. Three-dimensional reconstruction by zooming. IEEE Transactions on Robotics and Automation, 9(2): 196-207, 1993.

[3] S.-W. Shih, Y.-P. Hung, and W.4. Lin. Calibration of an active binocular head. IEEE Transactions on Systems, Man, and Cybernetics, 28(4):426442, 1998.

[4] K. Tarabanis, R. Y. Tsai, and D. S. Goodman. Calibration ^of a computer controlled robotic vision sensor with a zoom lens.

CVGIP: Image Understanding, 59(2):22&241, 1994.

[5] J. Weng, P. Cohen, and M. Hemiou. Camera calibration with distortion models and accuracy evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14( 10):965- 980, 1992.

[6] R. G. Willson. Modeling and Calibration of Automated Zoom Lenses. PhD thesis, Camegie Mellon University, Pittsburgh, Pennsylvania, Jan. 1994.

5. Conclusions

In this paper, we have proposed a motorized zoom lens calibration procedure for determining the relationship between the lens settings and the camera parameters. The major advantage of the proposed method is to reduce, in an adaptive manner, the amount of image acquisition and camera calibration, which is a time-consuming task, while maintaining the required calibration accuracy. In our exper-

498