Nonlinear minimization

The parameters estimated using linear algorithm such as Tsai's method are suggested to be tuned using a nonlinear optimization algorithm. The Levenberg-Marquardt algorithm (“LM”

for short) is a widely adopted optimizer for this purpose. The LM method approaches least-square curve fitting problem by a mixture of Gauss-Newton and gradient descent method. It convergences quickly even if the initial guess is not near optimum. When the estimate is close to optimal, its convergence slows down gradually.

Regarding a sum-of-square objective function  , the LM method tries to find a set of optimal parameters d ×1 that minimizes

=1

As a second-derivative optimizer, the LM method relies on the Jacobian matrix to find a direction to move. The Jacobian matrix referred in following context is defined in form

J =





For a d-dimensional optimization problem of a n-vector function the update of parameters is decided in each iteration by solving the normal equations

J^T J  I =−J^T , (3.29)

where J is the n-by-d Jacobian matrix,  is the damping scalar that controls the tendency of convergence, and the solution  is the step to next estimate. The square matrix J^TJ is the linear approximation of Hessian matrix, and is denoted by N. The update of current para-meters  is done by

^'= . (3.30)

The update of parameters is conditional. If new estimate introduces a worse outcome, the update is rejected. In that case, the damping factor  is multiplied by a predefined scale. On the other hand, it is decrease by the division of the same scale if a better situation is achieved.

The damping factor  dominates the behavior of the LM method. If  is set to a value significantly larger than the elements of N , then the optimization acts like gradient descent.

On the other hand, a very small value of  leads the LM method closer to the Gauss-Newton algorithm. As suggested by Marqardt, to avoid slow convergence in certain condition it is better to implement the modified version of (3.1)

[N  diag N ]=N^'=−J^T (3.31)

where the identify matrix is replaced by the diagonal of N .

In the application of optimizing perspective parameters, the function f is intuitively the

reprojection of n control points using parameters  , and yi−f_i  becomes reprojection error of i-th control point. The entry ^Jij of Jacobian matrix represents the change of repro-jected position of control point Oi in response to a small change of parameter j. Therefore the objective function  is the summation of squared reprojection errors. However, the hori-zontal and vertical errors are suggested to be dealt with separately [6]. The objective function therefore has a form of

where ^u and ^v are vectors of horizontal and vertical reprojection errors respectively. Also the Jacobian matrix is split into two parts

J =





An important issue when it comes to implementation is the computation of Jocobian matrix, which must be provided to construct normal equations. One way to obtain Jocobian matrix is by explicitly calculating the first-order partial derivative with respect to each para-meter in advance, whereas another approach is by numerical differentiation. The former is also known as symbolic differentiation approach. Both methods are implemented and tested.

In the experiments we found that the numerical differentiation has a better convergence rate over explicit calculation of Jocobian matrix. This leads a reasonable decision of applying numeral method in later experiments.

A problem arises when the optimization routine uses a 3-by-3 matrix to represent the rota-tion matrix. Treating the rotarota-tional part of the extrinsic parameters as nine independent elements is inappropriate since it has actually three degrees of freedom. One may fix this problem by augmenting the objective function with orthogonality and normality penalty terms. A more comprehensive solution is to instead utilize a proper parametrization of rota-tion. The rotation vector representation is implemented to avoid violation of rotational prop-erties during the optimization. The representation uses a 3-vector A to denote the rotation axis as well as the angle of rotation, which is determined by the norm of vector ∥A∥. The

conversion from a rotation matrix R to a its vector form A is done by

A= 

2sin r₃₂−r_23,r₁₃−r_31,r₂₁−r₁₂^T, (3.34) where the angle  is defined as

=arccos





Conversion of A back to matrix form R is attainable by

R=cos∥A∥⋅I₃sin∥A∥

∥A∥





The vector representation of rotation has a drawback of becoming numerical unstable under certain condition. The singular case happens as the angle of rotation approximates zero or multiples of . However, it is actually the case should be avoided in camera calibration.

When  is close to zero, the calibration plane will become nearly parallel to image plane. In that case the recovery of parameters is ill-conditioned, and the calibrated result is far from accurate.

OPTIMIZEPARAMETERS

Input: linearly estimated parameters Kc, R_c, T_c, camera-world correspondences ^IcO_c Output: optimized parameters Kc, R_c, T_c Fig. 3.5: Pseudo code of implemented camera optimizer

This algorithm is based on Levenberg-Marquardt method with numerical approximation.

3.4 Projector calibration

The unified calibration strategy calibrates projector like it is a camera with an exception that the correspondences between internal pixels and control points need to be provided by some projector-specific ways. As reviewed in previous chapter, a traditional projector calibra-tion approach relies on acquisicalibra-tion of sub-pixel correspondences. This seccalibra-tion presents an accurate, time-efficient, and robust way to calibrate a projector.