Robust Estimation of Parameters for Lucas-Kanade Algorithm

Robust Estimation of Parameters for

Lucas-Kanade Algorithm

Yih-Lon Lin1

Abstract The object tracking problem is an important research topic in computer vision. For real applications such as vehicle tracking and face tracking, there are many efficient and real-time algorithms. In this study, we will focus on the Lucas-Kanade (LK) algorithm for object tracking. In the standard LK method, sum of squared errors is used as the cost function, while least trimmed squares is adopted as the cost function in this study. The resulting estimator is robust against outliers caused by noises in the tracking process. Simulation is provided to show that the proposed algorithm outperforms the standard LK method in the sense that it is robust against the outliers in the object tracking problem.

Keywords Lucas-Kanade algorithm．Least trimmed squares

1 Introduction

The Lucas-Kanade (LK) algorithm was originally proposed by Lucas and Kanade in 1981 [1], which makes use of the spatial intensity gradient of the images to find a good match using a type of the Newton-Raphson iteration. The goal of the standard LK algorithm is to minimize the sum of squared errors (SSE) function between the template and the warped image region by adjusting the warping parameters.

Based on the theory of robust statistics, various types of cost functions were adopted in many pratical applications such as regression. One of the main approaches to robust regression involves M-estimation [2-6]. The M-estimator is

1 _{Yih-Lon Lin ()}

Department of Information Engineering, I-Shou University, Taiwan e-mail: yllin@isu.edu.tw

found to be robust to outliers in the response variable, but turned out not to be resistant to outliers in the explanatory variables, called leverage points. In fact, when there are outliers in the explanatory variables, the method has no advantage over least squares. In the 1980s, several alternatives to M-estimation were proposed as attempts to overcome the lack of such resistance. Least trimmed (sum of) squares (LTS) is a viable alternative [7-10].

There are several robust LK algorithms proposed in following literatures. The M-estimators were adopted to LK models for tracking problems in [11-15]. The basic idea is to replace the cost function SSE by Huber or Hampel function so that the effects of outliers may be degraded. In [11], the authors generalized the LK approach to histogram-based tracking algorithm. It establishes a closer link between template matching and histogram-based tracking methods. Schreiber [12] proposed a novel variant of the LK algorithm for tracking bilaterally symmetric planar objects from a moving platform. This algorithm was capable to cope with any warping transformation and can be generalized for the case of objects possessing higher symmetry. According to analysis of data distributions, Senst et al. [13] proposed a robust local optical flow approach based on a modified Hampel estimator with robust characteristics. Fan et al. [14] proposed a robust template tracking with weighted active drift correction. The minimization of active drift correction is achieved by the inverse compositional algorithm, which consists of the tracking term and the drift correction term. In [15], a robust LK template matching algorithm was based on evidence which is accumulated over many frames. This algorithm described the drift-correcting and used robust weights that are being updated from frame to frame. The previous papers emphasized robust inverse composition algorithm using a robust Huber or Hampel cost functions. However, there are many other methods for robust LK algorithm such as robust measurement of ocular torsion [16], visual tracking and learning [17], and graph-based transductive learning [18].

The LTS estimator is highly resistant to leverage points, and is robust to outliers in the response. When we expect there to be some number of observations in the data

that we wish to put no weight in the modeling, LTS estimator is usually a good choice. The percentage of the data that we wish to put no weight is termed the trimming percentage or trimming parameter, and this parameter is usually pre-specified for the data at hand. In this study, the main idea of this study is to utilize the resistant property of the LTS estimators in the LK algorithm (LTS-LK) for object tracking problems when there are outliers caused by, for instance, noises and occlusions. The updating rules for the warping parameters using LTS-LK approach will be derived. A simulation is provided in this paper, for book-object tracking. The simulation result shows that the proposed method can effectively track the object when salt-and-pepper noises corruption.

The rest of this paper is organized as follows. In Section 2, we outline the standard LK algorithm. Section 3 presents the detail of the proposed LTS-LK method. A experimental results is provided in Section 4 to verify the performances of the proposed method. Finally, a conclusion is made in Section 5.

2 Lucas-Kanade Algorithm

Template tracking of an object in a video sequence is performed by extracting a template in the first frame and then finding the region which matches the template as close as possible in the surplus frames. The goal of LK method is to align a template image T(x) to an input image I(x), where x[x ]yT is a column vector representing the pixel coordinates. Let W( px; ) denote the parameterized set of allowed warps, where p[p1p6]T is a vector of warping parameters.

Here, we consider the following set of affine warps

                             6 5 4 2 3 1 2 1 1 1 ) ; ( p p y x p p p p W W p x W . (2.1)

The best match to the template in the new frame is found by minimizing the following SSE function





2 ) ( )) ; ( (



  x x p x W T I E , (2.2)

where W( px; ) is defined in (2.1) and the sum is performed over all of the pixels

x in the template image T(x). Suppose a current estimate of p is known and we wish to compute an appropriate increment p. The minimization problem in (2.2) can then be converted to



  





2 ) ; ( min arg



   x p I Wxp p T x . (2.3)

Performing a first-order Taylor series expansion on I



W



x;pp





at p0 in (2.3) yields

 

 



_                   x x p p W p x W 2 ; I T I , (2.4)

where I



I x I y



T is the gradient of the image I evaluated at W ;

 

xp . The term Wp is the Jacobian of the warp. An approximate solution can be obtained by taking the partial derivatives of (2.4) with respect to p and setting them to zero. It is easy to derive that this approximate solution is given by

 



  







_             x x p x W p W p H I I T T ; 1 ,

where H is Gauss-Newton approximation to the Hessian matrix



                   x p W p W I I H T .

Finally, the parameters are updated as ppp, and the process is iterated until the estimate of parameters p converge or the pre-specified stopping criterion is met.

3 Robust Lucas-Kanade Algorithm

Let X n and Yp. Suppose we are given the training set







x d



X Y S l q q q    _ 1 , : .

In the following, we will use the subscript q to denote the qth observation. For instance, x_qi denotes the ith component of the qth input x_qn, ql, in.

desired output d_qk and the predicted output y_qk due to the qth observation is defined by qk qk qk d y e :  , ql, kp.

The LTS approach is to choose parameters that minimize the total sum of trimmed squared errors given by

 





 





       p k l q qk qk p k l q k q total e aRe e 1 1 2 2 1 * 1 2 2 1 2 1 ,

where l*l and the penalizing weight a

 

j is defined by

 

        . * , 0 *, 1 , 1 : l j l l j j a

Here, R

 

e_qk2 denotes the rank of the residual e2_qk, ql and e_{ }2₁k ...e_{ }2lk are

the ordered values of e1k2 , …, 2

lk

e . A popular choice of l* is

 

2





1



2



* l  n

l , where

 

b indicates the largest integer less than or equal to b. In the following, the term



ll*



l is referred to as the trimming percentage.

To address the object tracking problems when there are outliers caused by, for instance, noises, we will use sum of trimmed squared errors instead of the l₂-norm. Consequently, the goal of the LTS estimator is to find the parameters p of LK algorithm that minimize the cost function defined by

 



  











2



 

 

2 ; ;                      



p x p W p x W x p x W x T I I T I R a E .

It is not difficult to derive that the variable p can be calculated as

 



  

















W

 

xp

  

x



p W x p x W p x T I I T I R a H T T              



; ; 2 1 , where the robust Hessian matrix is

 



  















                    x p W p W x p x W T I I I R a H T T 2 ; .

-yes no   p End )) ; ( (W xp I ) ( x T ) ( x I I  W p  I 1  T H p 

Calculate steepest descent image

Calculate robust inverse Hessian matrix Calculate gradient image

Calculate p p p  Calculate penalizing weighta( j)

Fig. 3.1 Flowchart of LTS-LK algorithm.

4 Experimental Simulation

In this section, we compare the performances of standard LK and the LTS-LK algorithms for object tracking. Emphasis is put particularly on the robustness against outliers. Simulation programs are implemented using Borland C++ Builder 6.0 running on Microsoft Windows 7, Intel Core 2 Quad CPU, and 4-GB RAM platform.

To demonstrate the robustness of the proposed algorithm, we present an object-tracking experiments, consisting of one “book-tracking” video sequences. In Example 1, the frames after the 100th of the “book” sequence are corrupted by salt-and-pepper noises in which a corrupted pixel has the intensity of 0 or 255. The trimming percentage used in our simulations is 5%. In the simulation, we set 20 as the maximal number of iterations and 0.0001 as the minimal change of p. The

iteration stops when either of the two conditions is met.

Example 1. The "book" sequence is of size 320240 containing 250 frames in which the object to be tracked is the book. In this experiment, the initial bounding box size is 6080 pixels and contains about 6.25% of the frame size. Simulation results are shown in Fig. 4.1. The first and second rows show the results of the standard LK and the proposed LTS-LK algorithms, respectively. To demonstrate the robustness of LTS-LK algorithm, we randomly corrupt the video sequence with 20% noises, about 15,360 pixels, in the frames after the 100th frame. As observed, the standard LK algorithm fails to track the object in the 160th frame (the 3rd column), whereas our LTS-LK algorithm can effectively track the “book” object.

LK algorithm

LTS-LK algorithm

Frame 1 Frame 100 Frame 160 Frame 190

Fig. 4.1 Results of “book-tracking” sequence. Salt-and-pepper noises are added after Frame 100.

5 Conclusion

The main issue of this paper is the robustness against outliers caused by noises in the video object tracking problems. The proposed robust algorithm adopts the LTS scheme for the LK algorithm. Experimental results show that the proposed LTS-LK algorithm outperforms the standard LK. It is noted that SSE minimization criterion

is used in standard approach to search the optimal parameters. However, it is known that the estimation of parameters based on least squares criterion is vulnerable to outliers.

Acknowledgments The research reported here was supported by the National Science Council, Taiwan, under grant no. NSC 101-2221-E-214-074 and NSC 102-2221-E-214 -049.

References

1. Lucas, B. D. & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. in Proceedings of the International Joint Conference on Artificial Intelligence, Canada, pp. 674-679.

2. Tukey, J. W. (1996). A survey of sampling from contaminated distributions. in Contributions to

Probability and Statistics, Olkin, I., Ghurye, S., Hoeffding, Madow, G. W. W. G., Mann , H. B.,

Eds. Stanford Univ. Press, Stanford, California, pp. 448-485.

3. Huber, P. J. (1964). Robust estimation of a location parameter. Annals Math. Stat., 35, pp. 73-101.

4. Huber, P. J. & Ronchetti, E. M. (2009). Robust Statistics, 2nd ed. Hoboken, Wiley, New Jersey.

5. Hampel, F. (1968). Contributions to the Theory of Robust Estimation, PhD thesis, Univ. of California, Berkeley, California.

6. Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (2005). Robust Statistics: The

Approach Based on Influence Functions, John Wiley & Sons, New York.

7. Rousseeuw, P. J. (1985). Multivariate estimation with high breakdown point. in Mathematical

Statistics and Applications, Grossmann W., Pflug G., Vincze, I., Wertz, W., Eds. Dordrecht,

Holland, Reidel, pp. 283-297.

8. Tableman, M. (1994). The influence functions for the least trimmed squares and the least trimmed absolute deviations estimators. Statistics & Probability Letters, 19, pp. 329-337. 9. Giloni, A. & Padberg, M. (2002). Least trimmed squares regression, least median squares

regression, and mathematical programming. Mathematical and Computer Modelling, 35(9), pp. 1043-1060.

10. Nguyen, T. D., & Welsch R. (2010). Outlier detection and least trimmed squares approximation using semi-definite programming. Computational Statistics and Data Analysis, 54(12) pp. 3212-3226.

11. Schreiber, D. (2008). Generalizing the Lucas-Kanade algorithm for histogram-based tracking.

Pattern Recognition Letters, 29(7), pp. 852-861.

12. Schreiber, D. (2009). Incorporating symmetry into the Lucas-Kanade framework. Pattern

Recognition Letters, 30 (7), pp. 690-698.

13. Senst T., Eiselein V., & Sikora T. (2012). Robust local optical flow for feature tracking. IEEE

Transactions on Circuits and Systems for Video Technology, 22(9), pp. 1377-1387.

14. Fan, B., Du, Y., Zhu L., Sun J., & Tang, Y. (2011). A robust template tracking algorithm with weighted active drift correction. Pattern Recognition Letters, 32(9), pp. 1317-1327.

15. Fan, B., Du, Y., Zhu, L., Sun, J., & Tang Y. (2007). Robust template tracking with drift correction. Pattern Recognition Letters. 28(12), pp. 1483-1491.

16. Lee I. B., Choi B. H., & Park K. S. (2007). Robust measurement of ocular torsion using iterative Lucas-Kanade. Computer Methods and Programs in Biomedicine, 85(3), pp. 238-246.

17. Li, J., Wang, Y. & Wang, Y. (2012). Visual tracking and learning using speeded up robust features. Pattern Recognition Letters, 33 (16), pp. 2094-2101.

18. Zha, Y., Yang, Y., and Bi, D. (2010). Graph-based transductive learning for robust visual tracking. Pattern Recognition, 43(1), pp. 187-196, 2010.