Object Tracking

Object tracking has been one of the key processes in surveillance systems for a long time. Lots of researchers have put a great amount of efforts to develop techniques that can solve the correspondence problems. Establishing correspondence between frames has been thought as one of most challenging problems in object tracking, especially when occlusion occurs. Nowadays, due to the contributions of those researchers, the techniques of object tracking have evolved into much more complicated techniques. In this section, to obtain a comprehensive review of object tracking, we only introduce the basic form of some well-known tracking techniques rather than introducing the latest tracking techniques. The following discussion about object tracking is mainly based on [1].

In the following, we will introduce the basic methods for single-object tracking first. After that, we extend our discussion from single-object tracking to multiple-object tracking. We will introduce various approaches for multiple-object tracking. The main difference between single-object tracking and multiple-object tracking is the establishment of object correspondence over frames, especially when occlusion between objects occurs.

2.1.1. Single-Object Tracking

Single-object tracking is relatively simple compared to multiple-object tracking.

In single-object tracking, the needs for establishing correspondence between objects are much simpler. However, up to now, there is not any tracking technique that is suitable for all cases. Each tracking technique has its pros and cons. Different tracking

5

methods shall be adopted depending on the purpose and the circumstance. According to the approaches of these basic tracking techniques, we roughly classify them into two categories: deterministic tracking and probabilistic tracking.

2.1.1.1. Deterministic Tracking

The concept of deterministic tracking is quite intuitive. Given a region in the previous frame, we find the most similar region in the current frame. In this category, some algorithms take point-based approach, some use kernel-based approach, and others adopt silhouette approach. Below, we will introduce a few well-known tracking methods.

M

-

T

Mean-shift algorithm is a mathematic tool to find the local extreme value of any function. As long as an initial object appearance and a similarity metric are given, the mean-shift algorithm will maximize the appearance similarity iteratively based on the given similarity metric.

O

F

T

Optical flow tracking generates dense flow fields by computing the flow vector of each pixel under the brightness constancy constraint.

KLT T

As an interest point is given, KLT tracking iteratively computes the translation of given region centered on the interest point. Once the new location of the interest point is found, KLT tracking computes an affine transform to evaluate the quality of the new location of the interest point. If the quality is good, KLT tracking continuously

6

tracking this interest point and take its nearby region as a feature. Otherwise, the interest point is eliminated.

E

T

Eigen tracking builds a subspace representation of a given appearance by using Principal Component Analysis (PCA). Given an input frame, we reconstruct input appearance by using eigenvectors. The tracked object is found by minimizing the difference between reconstructed appearance and input appearance.

SVM T

In general, SVM is a classification scheme. As a set of positive and negative training features are given, SVM finds the best hyper-plane which separates these two classes with the largest separation. In SVM tracking approach, images of tracked objects as treated as positive features while things that are not tracked are treated as negative features.

C

T

Contour tracking methods are employed when tracking a complete object is needed. Starting at an initial contour, contour tracking methods iteratively evolve an initial contour to the current contour. In order to evolve the contour correctly, contour tracking methods require some degree of object overlapping between the previous frame and the current frame.

7

2.1.1.2. Probabilistic Tracking

In general, observations from surveillance video contain noise. Moreover, object motions can undergo random perturbation. Hence, contrary to deterministic tracking which uses observations only, probabilistic tracking tries to model the states behind the observations. In other words, probabilistic tracking use the state-space approach to take into account the observation uncertainties as well as model uncertainties. The state-space model of probabilistic tracking is shown in Figure 2-1. The states behind observations could be object position, velocity, and acceleration.

Figure 2-1 State-space model for probabilistic tracking

K

F

Kalman filter is one of the classical methods in probabilistic tracking. Kalman filter provides an efficient way to estimate the state which is governed by a linear stochastic equation and disturbed by an additive Gaussian noise. Not only the states are corrupted by an additive Gaussian noise, Kalman filter also assumes that the observations are perturbed by an additive Gaussian noise too. The assumption of additive Gaussian noises makes the closed-form solution is achievable.

Kalman filter is mainly composed of two steps: prediction step and correction step. Prediction step uses state model to predict new state, as shown in Figure 2-1.

Correction step uses the current observation to update the state to decrease the problem of error propagation.

8

P

F

However, the corrupted noises in observations and state models in reality would not be as simple as an additive Gaussian noise. Moreover, the state transition might not follow a linear dynamic equation. As a result, the closed-form solution will not be obtained easily or even be unobtainable.

Particle filter provides a solution for solving non Gaussian distribution and non-linear dynamic model by using a group of particle with different weights to approximate the distribution. Similar to Kalman filter, there are two steps in Particle filter, prediction step and correction step. Prediction step in Particle filter uses a set of particles to generate a new set of particles. Correction step in Particle filter uses observations to update the weight of each particle in new set. Since the objects are described as a probability distribution approximated by numbers of particles, we have a big chance to find the right position of object even when occlusion occurs.

2.1.2. Multiple Objects Tracking

Even though object tracking has been thought as a mature field, nowadays, a robust multiple objects tracking remains a challenging topic. One of crucial problems in multiple objects tracking is the establishment of correspondence. Unlike we discuss in single object tracking, we do not classify techniques into different categories here because most techniques we introduced in single object tracking cannot be applied directly in multiple objects tracking. Correspondence problem has to be solved first.

In the following, we will briefly describe some widely used methods that deal with correspondence problem.

9

J

P

D

A

F

(JPDAF)

The JPDAF [2], [3], is actually an extension of the Kalman filter. The JPDAF tries to associate all observations with each existing track in a probabilistic approach.

Rather than choosing the nearest neighbor or data closest to what is expected, the JPDAF computes new weights for the various candidate observations. Then, the JPDAF integrates these weights into the innovation step of Kalman filter. As a result, each observation is assigned to a certain track.

M

H

T

(MHT)

Since establishing correspondence with previous frame and current frame has a great chance to be inaccurate. The core idea of MHT algorithm is to defer the correspondence decision until enough observations are measured. The MHT keeps several correspondence hypotheses at each time step and over time period, the most likely hypothesis is chosen as the final track of object. With the help of deferring the time of making correspondence decision, MHT algorithm is able to continuously track multiple objects even some observations of objects are missing.

According to these basic tracking techniques above, we can find out that most of tracking techniques do not take the interactions between objects into account. Most of these techniques only consider the difference between objects and background and solve correspondence problem or collision problem as needed.