PDA Filter Based on the Winner-Update Strategy for Visual
Tracking
David Liu
1Li-Chen Fu
1,2Teng-Kai Kuo
1Jun-Wen Yeh
1Department of Electrical Engineering
1Department of Computer Science and Information Engineering
2National Taiwan University, Taipei, Taiwan, R.O.C.
Tel: (02) 23622209
Fax: (02) 23657887
E-mail: [email protected]
摘要
摘要
摘要
摘要
當影像複雜到無法正確分辨目標物與非目標 物時，PDA 提供㆒解決的辦法。以往 PDA 主要應 用在雷達㆖，本論文將 PDA 應用在可見光之影像 追蹤。由於使用可見光，目標物之形狀、顏色、 大小等特徵皆提供了更充裕的資訊。我們將此些 特徵與 PDA 做適當之結合，以提高追蹤成功率。 其㆗使用的影像辨識方法為 Winner-Update，此法 又較傳統之 SAD 節省 90%以㆖之運算量。 關鍵詞 關鍵詞關鍵詞 關鍵詞：PDA，Winner-Update，Kalman Filter，visual tracking。Abstract
The PDA filter deals with tracking a target when there is uncertainty in the origin of the measurements. Its use has primarily been on radar tracking tasks. While the target information is provided by a visible-light camera, much more information is available, like the shape, color, and size. We propose a method which incorporates this additional information into the PDA filter. The Winner-Update strategy is utilized for target detection and provides the likelihood ratio for the PDA filter.
Keywords: PDA, Winner-Update, Kalman Filter,
visual tracking.
1. Introduction
In a visual tracking problem the CCD camera provides the coordinates of each detected target. Measurements are affected by noise which is modeled as Gaussian. As the camera repeats capturing, the sequence of plots can be processed in a proper filter to smooth the measurement noise, thus providing target tracks. In the practical case, the problem is complicated by the false alarms owing to system noise and clutter. By clutter we refer to detections from nearby objects, weather, etc., that are generally random in location, intensity, and number. This may lead to several measurements in the “validation
region” [1] of a single target. The information of the target of interest is called the “target-originated measurement” in the Probabilistic Data Association (PDA) context. The PDA Filter [1,2] is a technique to handle the false data difficulty where the measurements are processed with a probabilistic weighting within the state estimation procedure.
In the visible light target tracking case, as opposed to the radar tracking case, much more target information is available, like shape and color. Traditionally, target detection is handled by techniques like the Sum of Squared Difference (SSD) [3], or by motion-based recognition [4], etc. The Winner-Update Strategy (WinUp) [5], which is a SAD-like template matching strategy, can save 91.6% to 98.0% of the absolute operations needed by the full scan SAD algorithm, depending on the image sequence.
The objective of this paper is to combine WinUp with the PDA Filter. In section 2 and 3, we briefly review the essence of WinUp and PDA respectively. In section 4 we propose how to produce a likelihood ratio and incorporate it into the PDA filter. Section 5 discusses the control method. Section 6 presents the experiment results.
2. The Winner-Update Strategy
The Winner-Update Strategy is a special case of the branch-and-bound strategy. The basic idea is that one does not have to examine all pixels within a block to find out which block has minimum error, since most blocks have very large error even with very little pixels examined. This leads to a huge reduction in computation time. Define the partial sum of absolute difference (PSAD) of l pixels, 2
,..., 2 , 1 B l= , as: | )) ( ), ( ( )) ( ), ( ( | ) , ( PSAD 1 1 0 y) (x, m j v y m i u x I m j y m i x I v u t l m t l + + + + − + + ≡ − − =
∑
where
{
(i(m),j(m))|m=0,...,B2−1}
is the index set of all the pixels in the block, and B is the block size. The index set determines the positions and the order of pixels in the matching block used for the accumulation of PSAD. Obviously, the followingholds true: ) SAD( ) , ( PSAD ) , ( PSAD ) , ( PSAD 2 B 2 1 u,v v u v u v u ≡ ≤ ≤ ≤
The last term, SAD (sum of absolute difference), is the most primitive template matching technique. It calculates the complete matching error, as opposed to the partial matching error PSAD. The Winner-Update Algorithm is summarized below:
The Winner-Update Algorithm
Given a template block at position (x,y) in It
begin
for each ( vu, ) in the search range do begin (initialization)
Calculate PSAD1(u,v)
PSAD(u,v):=PSAD1(u,v)
l(u,v):=1
end
select ( vuˆ,ˆ) having minimum PSAD(uˆ,vˆ) to be the temporary winner
while 2 ) ˆ , ˆ (u v B l < do begin l(uˆ,vˆ):=l(uˆ,vˆ)+1
calculate PSADl(uˆ,vˆ)(ˆ,ˆ)
v u PSAD(ˆ,ˆ): PSADl(uˆ,vˆ)(ˆ,ˆ)
v u v
u =
select ( vuˆ,ˆ) having minimum PSAD(uˆ,vˆ) to be the new temporary winner
end output ( vuˆ,ˆ)
end
3. The PDA Filter
The Probabilistic Data Association Filter (PDAF) [1,2] is used to handle the measurement origin uncertainty problem. It computes the posterior association probabilities for all current candidate measurements in a validation gate and uses them to form a weighted sum of innovations for updating the target’s state in a suitably modified version of the Kalman Filter.
3.1 State Space Model
The dynamics of the object are modeled by the equation
x(k+1)=Fx(k)+w(k) where
= ) ( ) ( ) ( ) ( ) ( ) ( ) ( k a k a k v k v k y k x k g g y x g y g x i i x
where i denotes the image frame, and g denotes the ground frame,
= 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 F
and the process noise vector is taken to be white Gaussian, with
E
{
w(j)w(k)'}
=δjkQThe measurement system is modelled as follows. If the measurement originates from the object in track, then
z(k)=Hx(k)+v(k) where = = 1 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 , ) ( ) ( ) ( ) ( ) ( H z k zv k zv k zy k zx k g g y x i i
and the measurement noise is taken to be white Gaussian, with
E
{
v(j)v(k)'}
=δjkR3.2 The PDA Implementation
The PDA filter is implemented by first performing measurement validation and then state estimate update, as summarized below.
3.2.1 The Validation Region
The measurements are reduced to a set of validated measurements by defining the following validation region
{
γ}
γ = + − + + ≤ +( ) : '( 1) ( 1) ( 1) ~ 1 1 k k k Vk zi νi S νi where νi(k+1)=zi(k+1)−zˆ(k+1|k) is theinnovation. Each measurement zi that lies within this region is considered validated. The threshold γ is obtained from tables of the chi-square distribution, since the weighted norm of the innovation that defines the validation region is chi-square distributed with number of degrees of freedom equal to the dimension
of the measurement. The computation of the measurement prediction zˆ(k+1|k) as well the state estimate are shown next.
3.2.2 State Estimation
Suppose at time k there are a number of mk
validated measurements. The set of validated measurements at time k is denoted by
{
}
mk i i k k Z 1 ) ( ) ( = z =and the cumulative set of measurements up to time k is
{
}
k j k j Z Z = ( ) =1Define the events θi(k)={zi(k) is the target
originated measurement }, i=1,...,mk, and
=
) (
0 k
θ { none of the measurements at time k is target-originated } with probability
{
k}
i i(k) P θ (k)|Z
β = , i=0,1,...,mk
The procedure that yields these probabilities is called PDA, and will be given in the next section. At the moment we assume they are known.
By the total probability theorem, the conditional mean of the state can be written as
{
}
{
} {
}
) ( ) | ( ˆ | ) ( ), ( | ) ( | ) ( ) | ( ˆ 0 0 k k k Z k P Z k k E Z k E k k i m i i k i m i k i k k k β θ θ∑
∑
= = = = = x x x xwhere xˆi(k|k) is the updated state estimate
conditioned on the event that the ith validated measurement is correct. This is given by the standard Kalman Filter as
xˆi(k|k)=xˆ(k|k−1)+W(k)νi(k)
where νi(k)=zi(k)−zˆ(k|k−1)is the corresponding
innovation, and W(k) is the standard Kalman gain. The error covariance associated with the updated state estimate is defined as
[
][
]
{
k}
Z k k k k k k k k| )= ( )−ˆ( | ) ( )−ˆ( | )′ ( x x x x Pand can be evaluated by ) ( ~ ) | ( )] ( 1 [ ) 1 | ( ) ( ) | ( c 0 0 k k k k k k k k k P P P P + − + − = β β where
)
'( ) ) ( ' ) ( ) ( ' ) ( ) ( ) ( ) ( ~ 1 k k k k k k k k k m i i i i W ν ν ν ν W P − =∑
= β and Pc(k|k)=(
I−W(k)H)
P(k|k−1).Now that we have obtained xˆi(k|k), it remains to
find out the values of βi(k).
3.2.3 The PDA Weights
The association probabilitiesβi(k)are developed in detail in [2] and require knowledge of the probability mass function of the number of false measurements (clutter). Assuming a Poisson density with parameterλVk, the weights are given by
∑
= + = k m i i i i e b e k 1 ) ( β , i=0,1,...,mk (1)∑
= + = k m i ei b b k 1 0( ) β (2) where − = '( ) − ( ) ( ) 2 1 exp k 1 k k ei νi S νi(
)
n k n(
D G)
D P P P c V b z z − = 2π γ 2λ 1 GP is the probability that the target-originated measurement falls within the validation gate, andPD is the probability that the correct measurement is detected, and cnzis the volume of the nzdimensional
unit hypersphere
(
c1=2,c2=π,c3 =4π 3,etc.)
.Fig. 1 Partition of the region of interest
4. PDAF with Winner-Update Strategy
To fully employ the image feature information of the target, we present the following method. The
(0,0)
(640,480)
x
y
A
B
C
D
region of interest (ROI) is partitioned into four parts, call them part A, B, C, and D (see fig. 1). The WinUp algorithm is then performed in each of the four parts. Each will then get a similarity value, SADi, with a smaller value indicating a higher likeliness. The similarity value is then converted into the likelihood ratio, defined by , 1,2,3,4 1 4 1 = + =
∑
= i i i i i SAD SAD λ .We can see that following this definition, the region with smaller SAD will have a larger likelihood ratio
λ. This likelihood ratio can be used to modify the standard PDA Filter’s association probabilities in equation (1),(2), as follows [6]: ) exp( ) exp( ) ( 1 i m i i i i i k k e b k e k λ λ β
∑
= + = , i=0,1,...,mk (3) ) exp( ) ( 1 0 i m i ei k b b k k λ β∑
= + = (4)where k≥0 is included to adjust the influence of the likelihood ratio on the standard PDA Filter’s association probabilities. If k=0, equation (3),(4) return back to their original form (eq.(1),(2)).
5. Motor Control
For the pan-tilt camera platform, the predicted target velocity in ground frame(vˆxg,vˆyg)and the
predicted target position in image frame(xˆi,yˆi) are
utilized for motor control. The control input is obtained by ⋅ + + ∆ ⋅ ∆ ⋅ = y error position image x error position image k t v t v f y input control x input control p y x g g u ˆ ˆ and ⋅ + ← y error position image x error position image kl u u
where u is initialized to zero. The function f is obtained by coarse calibration. The f term and the
p
k term constitute the basic control component based on the nominal system. As the nominal part would not represent the servomechanism exactly, additional enhancements to achieve effective control is required, and it is useful to consider an overall control law with the learning control [7] component u included.
6. Real Experiments and Results
The proposed method is demonstrated by a real indoor target tracking experiment. The target is an combat airplane model moving from left to right along a straight track, and the background is composed of one big poster with cluttered textures, and another stationary combat airplane model. These constitute the cluttered background and are used to compare the performance of the WinUp-PDAF with the standard Kalman Filter. The whole image is of size 640×480, and the block size B for template matching is 16×16
pixels. The search range for parts A, B, C, and D is
72
72× , and the overlapping length of two parts is 16 pixels, so that the total search range is 128×128 (fig. 1). For comparison, the WinUp-PDAF tracking module is compared with the standard Kalman Filter module. Both modules are using WinUp as the target detection method, while the Kalman Filter only keeps track of the point with minimum SAD. The template matching block is first locked onto the centroid of the target, and thereafter it is kept not updated.
Smooth control input is the desired goal. Since the smoother the control input, the less oscillation occurs on the camera, and this yields a higher confidence in target detection.
The results are shown in fig. 2(a~e) and table 1. Tracking with WinUp-PDAF yields smoother control input. This in turn yields higher confidence in target detection, since fast switching control input, as shown in fig. 2.(c),(d), results in large oscillation and serious image blurring. Even worse, the system will go lost of track as in fig. 2(e).
7. Acknowledgement
The Winner-Update algorithm is implemented in C language by the authors of [5] and downloadable at ftp://smart.iis.sinica.edu.tw.
Table 1. Results of the experiments Variance of control input (Unit in motor steps) Experiment 1 : Fig.2(a) WinUp-PDAF 100.61 Experiment 2 : Fig.2(b) WinUp-PDAF 115.49 Experiment 3 : Fig.2(c) WinUp-Kalman Filter 142.54 Experiment 4 : Fig.2(d) WinUp-Kalman Filter 173.22
130 140 150 160 170 180 190 200 210 220 230 -35 -30 -25 -20 -15 -10 -5 0 PDA1 time c o nt ro l i n pu t Fig. 2(a) 110 120 130 140 150 160 170 180 190 200 210 -35 -30 -25 -20 -15 -10 -5 0 PDA2 time c o nt ro l i n pu t Fig. 2(b) 40 60 80 100 120 140 160 -60 -50 -40 -30 -20 -10 0 10 NOPDA1 time c o nt ro l i n pu t Fig. 2(c) 40 60 80 100 120 140 -70 -60 -50 -40 -30 -20 -10 0 NOPDA2 time c o nt ro l i n pu t Fig. 2(d) 60 70 80 90 100 110 120 130 140 150 -40 -30 -20 -10 0 10 lost of track time c o nt ro l i n pu t Fig. 2(e)
Fig. 2 (a)~(e) : The horizontal ‘time’ axis has unit length equal to one cycle, which is about 120ms. The vertical ‘control input’ has unit length equal to one step of the stepping motor, while 10,000 steps correspond to 2π radians. Fig.2(a&b) are WinUp with PDAF. Fig.2(c&d) are WinUp with standard Kalman Filter. Fig.2(e) demonstrates a case where large oscillation finally lead to lost of track.
8. References
[1] Y. Bar-Shalom and E. Tse, “Tracking in a Cluttered Environment with Probabilistic Data Association,” Automatica, Vol. 11, pp. 451-460, 1975.
[2] Y. Bar-Shalom and T. E. Fortmann, Tracking and
Data Association, Academic Press, 1988.
[3] N. P. Papanikolopoulos and P. K. Khosla, “Adaptive Robotic Visual Tracking: Theory and Experiments,” IEEE Trans. Automatic Control, Vol. 38, no. 3, pp. 429-445, Mar. 1993.
[4] C. Cédra and M. Shah, “Motion-based recognition: a survey,” Image and Vision
Computing, Vol. 13, no. 2, pp. 129-155, Mar.
1995.
[5] Y. S. Chen, Y. P. Hung, and C. S. Fuh, “A Fast Block Matching Algorithm Based on the Winner-Update Strategy,” in Proceedings of the
Fourth Asian Conference on Computer Vision,
Vol. 2, pp. 977-982, Jan. 2000.
[6] D. Lerro and Y. Bar-Shalom, “Automatic Track Formation with Target Amplitude Information,” in Proceedings of Ocean Technologies and Opportunities in the Pacific for the 90’s, pp. 1460-1467, 1991.
[7] T. J. Jang, C. H. Choi, and H. S. Ahn, “Iterative learning control in feedback systems,”
