Fast block motion estimation using adaptive simulated annealing

(1)

Fast

Block

Motion Estimation Using Adaptive Simulated

Annealing

*

Mon-Chau Shie’, Wen-Hsien Fang’t, Kuo-Jui Hung2 and Feipei Lail

‘Dept. of Electrical Engineering

&

Dept. of Computer Science and Information Engineering

National Taiwan University, Taipei, Taiwan

Taipei, Taiwan

tTel:886-2-27376412

Fax :

886-2-27376424

Email

:

whf

@ e t . n t u s t . edu. t w

Dept. of

Electronic Engineering, National Taiwan University of Science and Technology

Abstract

This paper presents a simulated annealing (SA)- based algorithm for fast adaptive block motion estimation. The new algorithm begins with a set of checking points by exploiting the high correlations among the motion vectors of the temporally and spatially adjacent blocks. The SA algorithm is then em- ployed in the decision process to determine the next searching patterns. This new scheme provides a possibility of being able to move out of local minima so that the search process is less susceptible to such a dilemma. Additionally, an alternating search strat- egy is also addressed to visit more points without increasing computations. Simulation results show that the new algorithm offers superior performance with lower computational complexity compared with previous works.

1. Introduction

Motion estimation underlines the core of motion compensated predictive coding of image sequences. The block matching algorithms (BMA) in particular have received a great deal of attention and been adopted by various standards such as H.261, MPEG 1-2, and etc. [l].

The most straightforward BMA is the full search (FS) algorithm, which searches exhaustively over all allowable displaced points to locate the best match and thus requires enormous amount of computations. To mitigate this, various attempts have been made to reduce the number of search points while retaining acceptable image quality. For example, the popular three-step search algorithm (TSS) iteratively checks the surrounding nine points of the minimum with a *This work was supported by National Science Council of

R.O.C. under contracts NSC 87-2213-E011-014 and 87-2218- E002-020.

diminishing window size [2]. Several variants of the TSS such as the new three-step search (NTSS) algorithm [3] and the four-step search (FSS) algorithm [4] were addressed to determine a more precise motion vector with reduced computations. All of these fast search algorithms, however, are based on the implicit assumption that the block distortion measure (BDM) monotonically increases around the global minimum

[2]. Consequently, despite their efficiency, the search

is easily trapped into local minima, especially for high-activity image sequences.

Some approaches such as [5], which used the motion “flow” concept in determining the motion vector, have been addressed to refrain the search from being easily trapped into local minima. In this paper we propose a new algorithm by using the SA together with the local characteristics of the image. The new algorithm begins with a set of checking points by exploiting the high correlations among the motion vectors of the temporally and spatially adjacent blocks. We then invoke the

SA

algorithm in deciding the next search patterns. In contrast to previous works which are “greedy” in the search process, the new scheme possesses a mechanism ofjumping out of local minima and thus is less susceptible to this dilemma. Further- more, alternating checking point patterns for adjacent blocks are used so that the search can visit more points without increasing computations. As a consequence, the motion vectors can be accurately determined with lower computational overhead as compared with previous works. The furnished simula- tions justify this new algorithm.

2. Simulated Annealing-Based

Algorithm for Motion Estimation

In this section, we describe an SA-based algorithm for fast adaptive motion estimation. Before address-

(2)

ing the proposed algorithm, we briefly review the SA.

2.1 Simulated Annealing

The SA is a well-known effective technique for solv- ing iterative optimization problems and has been suc- cessfully applied to the contexts of circuit design, and vector quantization [6]. In such problems, the tradi- tional iterative search process is easily trapped into local minima on the way to find the global minimum. In contrast, the SA allows the possibility of escap- ing from local minima by providing a mechanism of uphill move according to a stochastic decision rule. More specifically, the SA is constituted by the fol- lowing two rules in the transition of “states” :

1. A change of state which induces a reductian of the associated energy is always allowed. 2. If a change of state induces an increase of the

associated energy, the change is then accepted with the probability

-(E - E )

p=e--+--l-

where E1 and Ea are the associated energy of the present and next states, respectively, T is a control parameter (called “temperature”), and

IC

is the Boltz- mann constant. As a consequence of (l), we can note that the scheme always takes a downhill step (reduction of energy) in a transition of states, and some- times an uphill step (increase of energy) with probability P , which decreases as the energy difference increases. It is this uphill climbing technique which enables the search process to be more robust against local minima, Also, the temperature T decreases as

iterations proceed. This reflects the fact that it is less likely to be trapped into local minima as the search gets closer to the global minimum. A simple mech-

anism to adjust T is to reduce it by a ratio in each iteration.

2.2 Proposed Algorithm

The proposed algorithm begins with an appropri- ate choice of checking points. Due to the fact that most image sequences only involve gentle movements, there exist high correlations among the motion vectors of the temporally and spatially adjacent blocks. As such, we adopt the center of the search window of the present block and the motion vectors of the spatially and temporally adjacent blocks as well as their neighboring points as our starting points to locate the best match. More specifically, these starting candi- date points include

CA,

Cif-1, and

e:-’,

where

CA

denotes the checking point pattern that includes the center located at the present block and its neighboring points, whereas Cfel,and

Cjf-l,

denote the

. . . .. . .

i- I

1

i t 1

tth

frame

(t-1)” frame

Figure 1: The starting checking point patterns of

C ~ , C ~ - l , C ~ t _ l ,

and

e:-’.

checking point patterns based on the motion vectors and their neighboring points of the two spatially adjacent blocks and those of the corresponding block of the previous frame as shown in Fig. 1. Due to the fact that the center-biased characteristic of the image sequences as addressed above, most of these checking points are in common and therefore do not incur too many computations.

In order to avoid the search to be trapped into local minima, we invoke the SA algorithm in the decision of the next search patterns. To apply the SA algorithm in this problem, the state denotes the checking point pattern and the energy associated with the state corresponds to the minimum BDM based on these checking points. Also, we adopt a normalized T as T

=

so that the same probability measure (1) can be used for every block.

We start the SA by choosing the energy of the ini- tial state as the minimum BDM, E l , based on the aforementioned starting checking points and the en- ergy of the next state, E2, as

E2

=

min{b:-,,

bi-l,

bk-l} (2) where

bf-l,bf-l,bh-l

denote the minimum BDM of the two spatially and temporally adjacent blocks based on the optimum motion vectors, respectively, and have been previously determined.

If E2

<

E l , we expect that the present block in-

volves fast movement and the next search pattern should be closer to the boundary of the search window. We then use a larger search pattern C d , where d denotes the corresponding search size, and deter- mines the minimum BDM based on these new check- ing points as the new E2. If the new E2 is still smaller

than E l , we shrink the size of the search pattern as Cj. and repeat the above steps. On the other hand,

(3)

if E2

>

El, we use the SA algorithm to determine whether we should proceed the same steps as the above or we should directly go to the final fine search. Such a procedure repeats until the minimum search window is attained. The final fine search is to check the optimum motion vector determined so far along with its nine neighboring points, C e n d , as shown in

Figs. 2 and 3, to locate more accurately the best match.

In addition, in order to reduce computational overhead while still visiting enough checking points, we use alternating search technique between two adjacent blocks, as shown in Figs. 2 and 3, respectively, with tu

=

7. The overall procedures for the proposed algorithm can be summarized as (the mean absolute difference (MAD) is used as the BDM):

1. (Initialization) Let

El=

minimum MAD based on the checking point patterns of Ch, C!-l, C!-l, and

Ci-’,

Ez= minimum of {bj-l, bj-l,bk-l}, the search window size=(2w+l) x (2w+1), i = 1. 2.

DO

SA(E1, Ea, T,ACCEPT). If ACCEPT=O, go to Step 3; Otherwise, d

=

log(%), E2= minimum based on the checking point patterns {Cd}, T = T x

p ( p

is a reduction ratio), i = i

+

1, repeat Step 2.

3. Perform the final checking point pattern C e n d .

4. Switch the search patterns for the next block and go to Step 1.

where

Procedure SA(E1, E2,

T ,

ACCEPT) If E2

<

El, ACCEPT=l;

Otherwise,

ACCEPT=l with probability P as given in (1).

3. Simulation

Results

In this section, some simulation results are pro- vided to verify the proposed algorithm. The test image sequences include the “Salesman”, “Flower”, “Table tennis”, and “Football” image sequences. Four similar algorithms, including the FS, TSS, NTSS, and

FSS,

along with the proposed algorithm (with the parameters ,O = 0.8 and L

=

O . l ) , have been carried out for comparison.

The resulting average mean squares error per pixel (MSE/pixel) and the average search points per block for the first 90 frames of the test image sequences us-

ing these algorithms based on H.261 with M

=

N

=

16 and w

=

7 are listed in Table 1. As a vivid illustra- tion, the MSE/pixel vs. the frame number for the test image sequences “Table tennis” and “Football” are also shown in Figs. 4 and 5, respectively (for clarity,

only the results from the 21S* to the 60** frames are shown). FromTable 1, we can note that the proposed algorithm outperforms these four algorithms by providing a smaller MSE/pixel (except the FS) with even lower computational complexity. The small MSE’s of the proposed algorithm is due to the full exploita- tion of the temporal and spatial relationship among the motion fields, and the SA algorithm. The lat- ter also explains that the proposed algorithm works in particular well for high-activity image sequences such as the “Flower” and “Football” sequences, for which the search using the previous fast algorithms is easily trapped into local minima.

4.

Conclusions

In this paper, we describe a new BMA for fast motion estimation. The algorithm uses high correlations among the motion fields of the spatially and temporally adjacent blocks, the SA in the transition of search patterns, and the alternating search technique for adjacent blocks, to attain superior performance with lower computations compared to previous approaches. Consequently, it offers an appealing alter- native for the BMA for motion estimation.

References

[l] A. M. Tekalp, Digital Video Processing, Prentice-Hall, Englewood Cliffs, NJ, 1995. [2] T. Koga, K. Iinuma,

A .

Hirano, Y. Iijima, and T .

Ishiguro, “Motion-compensated interframe coding for video conferencing,” in Proc. N T C 81, pp.

C9.6.1-9.6.5, New Orleans, LA, Nov./Dec. 1981. [3] R. Li, B. Zeng, and M. L. Liou, “A new three-

step search algorithm for block motion estimation,” IEEE Trans. Circuits Syst. for Video

Technol., vol. 4, pp. 438-442, Aug. 1994.

[4] L.-M. Po and W.-C. Ma, “A novel four-step

search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. for Video Technol., vol. 6, pp. 313-317, June 1996. [5] Jai-Shin Jan, Wen-Hsien Fang and Ming-Yi Yu

“An adaptive flow-based dynamic search algorithm for block motion estimation,” in Proc. Int’l Symp. Circuits and Sys., pp. 2092-2095, Hong Kong, June 1997.

[6] P.J.M. Van Laarhoven and E.H.L. Aarts, Sim-

ulated Annealing: T h e o r y and Applications,

Kluwer Academic Publishers, Dordrecht, Hol- land, 1987.

(4)

Figure 2: The checking point patterns of

Cc

Cj-l,

CjW1,

Ci-',

C d , and

E,,,;

used in this paper with w

=

7 Temb Frame r --I

-

TSS NTSS

-

FSS

Figure 4: Comparison of MSE/pixel vs. frame number for various algorithms based on the Table Tennis sequence.

Figure 3: The alternating checking point patterns of Fig 2. FootLM Frame 2 I 20 25 30 35 40 45 50 85 F r m e Number

Figure 5: Comparison of MSE/pixel vs. frame number for various algorithms based on the Football sequence.

Sales. Flower T . Tennis Football Sales. Flower T . Tennis Football

FS 27.29 269.15 140.25 384.88 FS 204.28 202.05 202.05 202.05

TSS 28.15 315.49 174.92 416.43 TSS 23.24 23.25 23.14 23.09

NTSS

27.40 277.64 161.06 412.53

NTSS

16.75 21.49 19.72 20.57 '

FSS 27.93 292.23 159.69 428.89 FSS 16.21 18.89 17.94 18.04

Proposed 27.36 272.57 150.35 405.93 proposed 13.19 17.83 16.60 17.62

Table 2: Comparison of average number of search points per block for various algorithms based on the first 90 frames of the test image sequences.

Table 1: Comparison of MSE/pixel for various algorithms based on the first 90 frames of the test image sequences.