Wavelet-based active contour model for object tracking

(1)

WAVELET-BASED ACTIVE CONTOUR MODEL FOR OBJECT TRACKING

Jen-Chang Liutt, Wen-Liang Hwangt , Ming-Syan Cheni, Jin- Wu Tsai, and Chi-Hung Lin

Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan4

Institute

of

Information Science, Academia Sinica, Nankang, Taipei, Taiwant

Institute of Microbiology and Immunology, National Yang-Ming University, Taipei, Taiwan*

*Email

:

whwang@iis.sinica.edu.tw

tFax:

886-2-27824814

ABSTRACT

We propose an integrated wavelet-based framework for active contour model(snake), which is used for motion tracking of deformable objects in a sequence of images. The input image frame is decomposed into multi-resolution representation using wavelet transform. First, the wavelet transform coefficients are used in a multi-resolution motion estimation process to find the initial contour in the frame. The presented multi-resolution motion estimation method allows larger movement of the tracked object than the traditional image- based motion estimation. Secondly, the wavelet transform modulus at each scale are considered in the energy function of the active contour model. The application of biological cell tracking using the proposed method is shown as an example.

1. INTRODUCTION

The problems of object segmentation and object tracking are becoming more important in the recent years, because of the development in the advanced video compression standard, such as MPEG-4 and MPEG-7. Many algorithms are proposed to solve the above problems. Among the proposed methods in the literature for these problems, we are inter- ested in the active contour model(snake) [ 11 [7] because it can solve the segmentation and tracking problems simultaneously [2], and it involves high-level visual knowledge as an aid by an interactive process.

The snake is basically a energy-minimization model, which seeks to minimize the energy of the contour spline in the image, which includes the energy caused by the external forces, internal forces, and other constraint forces. Once a reasonably good initial contour is chosen, a hierarchical filtering method, also known as the continuation method [ 2 ] [ 3 ] , is often used to find the best solution of the snake. The continuation method is a process which tracks the best solution from coarse to fine scales in a scale-space representation of the image. Our first innovation extends the idea of the continuation method to the wavelet-based processing of the active contour model.

0-7803-6725- 1/0 1/$10.00 0200 1 IEEE 206

The result of snake in the previous frame can be used as an initial contour in the current frame for object tracking [ 2 ] .

If the object movement is large from the previous frame to the current frame, this estimated initial contour is worse and the resulting snake will also be worse. Our second innovationis to apply the multi-resolutionmotion estimation (h4RME)[4] to estimate a good initial contour for the current frame. The presented MRME allows larger movement of the tracked object than the traditional image-based motion estimation. With the above innovations, we have an integrated wavelet- based framework for active contour model, as shown in Fig.

1. The details will be explained in the following sections. snake I I I I contour calculation wavelet Fig. 1. contour model.

Flow chart of the proposed wavelet-based active

2. THE WAVELET-BASED ACTIVE CONTOUR

MODEL

In this section, we first review the snake model and the continuation method. Then we explain our proposed wavelet- based active contour model.

2.1. Snake and the continuation method

The snake describes a deformable curve

w

( s

,

t

) , with param- eter s for spatial index and

t

for time index. In the image plane, this deformable curve is a function of the coordinate variables 5 and y.

(2)

where

s2

and T are defined open intervals. function of the snake is defined as [ 11 [2]

The energy

where E.i,,t is the internal potential force of the snake, Eezt is the external force of the snake caused by the image, and

E,,,, is the additional constraint forces. Details of the formulas and designs of these forces can be found in [ 11 [2] [7 ] . The snake model is a energy minimization process which calculates a curve w*(s) with the minimal energy E ( v * ) .

The key that the calculated curve w*(s) is the object contour, depends on the proper design of the image force E e Z t .

As an example, finding edge contour in an image can be done by setting Eert = - I V I ( x , y)l, where

I(x,

y) is the intensity of the image. The snake is attracted to contours with large image gradients, which correspond to possible image edges.

Because most images contain a lot of noise, applying the snake model directly to the images often produces unstable results. The idea of continuation method is that we calculate the snake in a coarsely smoothed image, and the result of the snake at the coarse scale is take as an initial snake in a finely smoothed image to compute a fine-tuned snake, and this process is repeated. The implementation of the continuation method in [2] is illustrated as in Fig. 2.

original

Gaussian smoothing

.I

-

scale

fine coarse

Fig. 2. Flow chart of the continuation method

The original image is filtered through a family of Gaus- sian filters with different resolutions. It can be viewed as a

method to produce a discrete version of the scale-space representation. A differentiation filter, such as the Sobel filter, is then applied to these Gaussian filtered images, to produce approximations of the gradient of the Gaussian smoothing images. The continuation method starts from the gradient of Gaussian image at the coarsest scale. After the snake in the coarsest scale is found, it is taken as an initial contour for the following finer scale. The process goes on until the finest scale is reached.

2.2. Design of the wavelet-based snake

The multi-scale structure of the continuation method is natural in the world of wavelet transform. In fact, it has been shown [ 51 that the first derivatives of a well-defined family of Gaussian filters are equivalent to the corresponding wavelet transform coefficients multiplied by a scaling constant. We show the 1-D case in the following as an explanation. For

a smoothing function B(s), such as a Gaussian filter, we suppose that O ( x ) is differentiable and its first derivative is denoted as

$(x)

=

dO(x)/dx.

By definition, the function

$(x)

is considered to be a wavelet because it satisfies the following condition:

+ W

$(z)dx

= 0. (3)

L

Let's denote the dilation of any function

E

by a factor s as (4)

The wavelet transform of

f(x)

at scale s can be written as

w s f ( . ) =

f

*

)-s(.$ (5) We derive that

We can derive similar formulas for 2-D case, where the x- and y-directional partial derivatives are used instead.

From the above derivation, we know that the filtering part of continuation method is equivalent to the wavelet transform if the family of the Gaussian filters is proper chosen to satisfy the 2-D dilation equation

1 X Y

5-2 s ' s

O,(x,y) = - O ( - '). ( 7 ) We define the 2-D wavelet functions as

then the wavelet transform of image I ( % , y) at scale s is defined as

w:I(x,y) = r*$;(x,y),Tv:I(x,y) = r*$;(x,y). (9)

It has been shown [ 5 ] that

Therefore, we define the external energy Eezt at scale s as the modulus of the wavelet transform at scale s

The architecture of our proposed wavelet-based snake model is shown in Fig. 3. Our proposed model involves the theory of wavelet transform as a solid background for analysis purpose and is more flexible in the choice of filters.

(3)

* - - ~ T L - - - l p z - ~ ~ y ~

final calculation calculation calculation snake

-

* scale

tine cotuse

Fig. 3. model.

Flow chart of the wavelet-based active contour

3. INITIAL CONTOUR ESTIMATION USING MULTI-RESOLUTION MOTION ESTIMATION Given a good initial contour, the snake will converge to the object boundary in an image according to the energy mini- mizationprocess. In a sequence of images, the target object might have global movement and local deformations. As- sume that the object can move only a small distance or have small deformations between frames, the resulting snake contour at frame

(i

- 1) can be used as a reasonable good initial contour at frame

i

and the object segmentation at frame i can be solved accordingly. Therefore, the object segmentation and tracking can be solved simultaneously by the active contour model. However, the constraint of small motion is too restrictive to apply to all kinds of image sequences. Thus motion estimation for the initial contour at each frame is necessary when the object movement is large between frames.

The goal of motion estimation in our model is to match the snake contour points v [ s , i - 11 at frame

(i

- 1) to the initial estimated contour points v[s,

i]

at frame

i.

Lots of motion estimation methods are proposed [6]. Among them, the block-based method is commonly used, such as the full search, three-step search, hierarchical motion estimation and multi-resolution motion estimation(MRME) [4]. The full search is optimal but inefficient, while the others are suboptimal methods and provide different computational complexity and accuracy. Since we haveused wavelet transform in the active contour model, it is natural to adopt the MRME for motion estimation in our design.

We briefly overview the idea of MRME [4]. Fig. 4(a) shows the subband structure of wavelet transform. The LL, LH, HL, and HH represent the combination of low-pass and high-pass filtering in the x- and y- direction respectively, and their subscripts denote the indices in the scale-space. Fig. 4(b) shows the motion vectors in the subbands. Note that the motion vectors in the same subband orientation at different scales are not related by simple scaling due to the effects of down-sampling and the non-shift-invariant nature of wavelet transform. MRME mainly explores the correla-

Fig. 4. (a) Subband structure of wavelet transform. (b) Multi-resolution motion estimation on (a).

tion of motion vectors at different scales. Therefore MRME starts from the coarsest subbands with large search region, and the motion vectors are propagated through the finer subbands, and are refined with smaller search regions.

We implemented a modified version of MRME. Since we are concerned with the object boundary only, the wavelet transform modulus at each scale are used instead of the all wavelet transform subbands. We start estimate the motion vector from the coarsest LL subband with large search area which allows large object motion, then this motion vector is propagated through wavelet transform at finer scales and is refined by searching the wavelet modulus image with smaller search area around the motion vector. Finally this motion vector is refined in the original image. Our modified MRME

algorithm is shown in Fig. 5.

-

scdc coarse fine

Fig. 5. Flow chart of the modified multi-resolution motion estimation.

4. APPLICATION TO THE BIOLOGICAL CELL TRACKING

In biological science, it is interesting to understand information about the living cell movement, such as the shape deformation, dynamics, and path of motion. Quantification of related data is required but impractical for human to do such a job. Automatic approach to track and analyze the cell motion is required. Several methods are proposed to do this work [ 2 ] .

(4)

We applied our proposed wavelet-based active contour model to the biological cell tracking problem. Fig. 6 shows the image of anliving cell and its wavelet transformmodulus at different scales. The image is smoother at coarser scale, thus noise is smoothed and the object boundary can be seen clearly

(a) Original image (b) scale 2l

(c) scale 22 (d) scale 23

Fig. 6. (a) Original image. (b)-(d) Wavelet transform mod- ulus at scale 2’,2’, and 23.

The calculation of energy minimization is based on the discretization version of (2). The terminating condition for snake convergence is based on the steady-support criterion

,

which seeks to minimize the averaged energy Ele,Lgtlb(w)

along the snake [2]. Fig. 7(a) shows the snake contour found at scale 4. Fig. 7(b) shows the plot of ElelLgtlL(w)

as a function of the number of iterations. Fig. 8 shows the results of tracking a cell with large movement in an image sequence of 20 frames.

Fig.

7.

(a)’o’:initial contour; ’+’:converged snake contour.

@>

Plot Of E l e n g t l l (VI.

Fig. 8. (a)-(c) Extracted images from an image sequence of

20 frames. (d) The trace of cell contour.

5. REFERENCES

M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contourmodels”, Proc. First Int. Con5 Computer Vision, pp. 259-268, 1987.

F. Leymarie and M. D. Levine, “Tracking deformable objects in the plane using an active contour model”, IEEE Trans. on PAMI, pp 617434,1993,

A. Witkin, D. Terzopoulos, andM. Kass, “Signal match- ing through scale space”, Proc. 5th Nat. Con5 Artificial Intell., pp 714-719, 1986.

S. Zafar, Y.-Q. Zhang, and B. Jabbari, “Multiscale video representation using multiresolution motion compensa- tion and wavelet decomposition”, IEEE .I.on Selected Areas in Comm., pp 2432,1993.

S. Mallat and S. Zhong, “Characterization of Signals from Multiscale Edgets”, IEEE Trans. on PAMI, pp 710-732,1992.

A. Murat Tekalp, “Digital video processing”, Prentice Hall. 1995.

H.-H. Wu, J.-C. Liu, and C. Chui, “A wavelet-frame based image force model for active contouring algorithm”, IEEE Tram. on Image Processing, vol. 9, no.

11, Nov. 2000.

Wavelet-based active contour model for object tracking