AN ADJUSTABLE BLOCK MOTION ESTIMATION ALGORITHM BY MULTIPATH SEARCH

全文

(1)AN ADJUSTABLE BLOCK MOTION ESTIMATION ALGORITHM BY MULTIPATH SEARCH Thou-Ho (Chou-Ho) Chen. Chou-Yu Chen. Department of Electronic Engineering, National. Department of Electronic Engineering, National. Kaohsiung University of Applied Sciences. Kaohsiung University of Applied Sciences. thouho@cc.kuas.edu.tw. 1093320122@cc.kuas.edu.tw. ABSTRACT. exploiting the heavy temporary redundancy between successive frames. Among various motion estimation. A speed-quality adjustable block-matching algorithm. techniques, the block-matching algorithm (BMA) is. (BMA) based on multipath flatted-hexagon search. the. (MFHS) is developed for motion estimation. To. international video compression standards including. improve the accuracy of fast BMA near to that of full. H.261, H.263, H.264, MPEG-1, MPEG-2 and. search (FS), the problem of being trapped at the local. MPEG-4. minimum block distortion measure (BDM) should be. effectiveness and simplicity for implementations.. overcome. In the proposed method, an adaptive. However, the matching process of finding the optimal. threshold of BDM is introduced to determine the. still involves a large amount of calculations, e.x., the. required search directions in order to escape from. most accurate approach, called the full search (FS). being trapped at a minimum BDM, followed by a. method which requires to evaluate all candidate. flatted-hexagon search performed in the direction. blocks,. according to the BDM below a threshold. Then, the. computational power for the MPEG-4 encoder. To. motion vector will be refined at each search step. reduce the intensive computational complexity with a. unite the searching process is stopped. The BDM. tolerable. threshold will be adaptive for the purpose of. algorithms were developed [1] [2] [7]-[15].. most. attractive. [3]-[5]. can. method. [16]. consume. distortion,. for. [17],. at. many. the. because. least. fast. 63%. current. of. of. its. the. block-matching. adjusting the search speed and matching probability required in the specific application. Experimental. Among the above suboptimal methods, both the. results show that the proposed MFHS algorithm can. search pattern’s attributes and initial searching range. achieve a matching probability upto 99.35% and. always directs the developmental processes of these. 13.85 times of search speed of FS in some certain. algorithms. By taking advantage of the characteristics. sequence.. of the center-biased motion vector distribution. Keyword: Multipath search.. existed in most real-world image sequences, the new three-step search (N3SS) [11], four-step search (4SS). 1. INTRODUCTION. [10] and block-based gradient descent search (BBGDS) [9] perform better than the three-step. Motion estimation can make the interframe. search (3SS) [15], where these four search patterns. coding to achieve a very high compression ratio,. are. when compared to the intraframe coding, by. compact-shaped pattern with fewer candidate search 1. square-shaped.. Based. on. a. practical.

(2) points per block, a diamond-search (DS) algorithm. To achieve a matching accuracy near to FS with. [7][12] can not only improve the searching speed but. a larger search speed than FS, this paper introduces a. also reduce the chances of being trapped in local. multipath flatted hexagon search (MFHS) strategy. It. optimal, when compared to those four algorithms.. can effectively solve the problem which is trapped at. The hexagon-based search (HEXBS) algorithm [2]. local optimal to improve matching the probability.. utilized a hexagon-shaped pattern with only 7. And, it can adjust the searching speed and matching. checking points in the initial search and 3 checking. probability for your requirements.. points in the following searches to achieve substantial. 2. ANALYSIS OF THE SEARCHING STRATEGY. speed improvement over the DS algorithm with similar. distortion. performance. for. most. high-resolution (e.x., 720×480) image sequences. Nevertheless, the matching-probability performance. Almost. will degenerate with the decreasing resolution of the. algorithms are explicitly or implicitly based on the. video format. To obtain a faster searching speed than. assumption: BDM increases monotonically as the. the DS algorithm while maintaining similar search. checking point moves away from the global. quality, the cross-diamond search (CDS) algorithm [1]. minimum. Obviously, this assumption essentially. employed a cross search pattern at the initial step to. requires that the error surface is unimodal over the. exploit the characteristics of the center-biased motion. search windows. Unfortunately, this is usually not. vector distribution very efficient, followed by the. true due to many reasons such as the aperture. halfway-stop technique, and the large/small diamond. problem, the textured (periodical) local image content,. search patterns in the subsequent steps. Although,. the inconsistent block segmentation of moving object. various search patterns and processes at different. and background, the luminance change between. steps will make the CDS algorithm to be complicated. frames, and etc. As a consequence, the search would. on realization, especially for VLSI implementation. easily be trapped at a local minimum.. due to its favoritism of regularity [14]. Flatted. -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7. hexagon searching (FHS) [13] method used the simple strategy which is different from CDS and solves the problem which is not enough accuracy of HEXBS. The covering range of a search pattern should be enlarged as horizontal as possible to find the optimal motion vector quickly because the probability of horizontal-biased motions is larger than that of vertical-biased motions in most of the real-world image sequences. However, those previous fast BMAs can’t achieve a similar matching probability as that of FS under a moderate search speed, especially for applications in which a higher. conventional. block. motion. estimation. -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7. 4. 68 435. 345 345. 67. 345 234. 234 234. accuracy of motion estimation is required. Fig. 1 The example of false searching for direction 2.

(3) described as follows:. The general fast searching methods use search pattern to search the direction of a minimum BDM, and close step by step in minimum BDM. But, they. M. N. SAD (i, j ) = ∑ ∑ f ( m, n) − f ′(m + i, n + j ). can cause the faulty searching direction due to all. (1). m =1 n =1. above-mentioned reasons while beginning to search BDM, especially they have two or more minimum. (i, j ) is a candidate motion vector. f (m, n) is the. BDM. When the searching direction is faulty, they. pixel intensity at ( m, n) of the present frame.. usually difficult to go back correct direction and then. f ′(m + i, n + j ). they are trapped at a local minimum. Fig. 1 is the. ( m + i, n + j ) of the reference frame. For solving. example which is the faulty searching direction of DS.. the problem of searching the faulty direction, we. Among the example, the value of a circle is the BDM. propose a multi-path search algorithm. We define a. value which is calculated. The charcoal gray circle. function as eq. 2. SADcurrent is the SAD value of. (-2,-5) is the location of a true motion vector. DS. every new checkpoint of a search pattern. SADmin is. calculate first 9 checking points, and the minimum. the minimum SAD value for all checked points, and. value which is located at (2,0) is 67. Then, DS is. T is a threshold value. If the correlation of SADcurrent ,. based on this point to build a new diamond pattern.. SADmin and T accords with eq. 2, we will build a. So, the search pattern goes forward from the direction. new search pattern whose center is the location of. of that black arrow. But, the minimum BDM of the. SADcurrent . If not, we stop the algorithm.. is. the. pixel. intensity. at. example is located at (-2,-5). As a result, the example should be gone forward the direction of the second. SADcurrent - SADmin ≤ T. minimum BDM, namely the direction of the gray. (2). arrow is correct. We often ignore the importance of the second minimum value. In fig.1, if we can search. 3.2 Dynamic threshold. the directions of the first and second minimum values at the same time, the faulty matching probability can. The same value of T can cause some different. be to decrease. However, when should we to search. effective results on various image sequences due to. the direction of the second minimum value?. various video features. As a result, we present a. Whether the direction of the third minimum value. scheme which is adaptive adjust threshold for various. also should to search or not? We should compare the. video contents and features.. BDMs of the points of all directions, and we judge whether the values and minimum BDM are similar. If. T = SADmin × β. yes, we will search the points. This guides the basic. (3). idea of our adjustable fast motion estimation. After our test and verification, T can make the. 3. METHODOLOGY. adjustability multi-path search method speed and accurate rate better through simple processing, and the adaptability is stronger. The β value, which is. 3.1 Multi-path search algorithm. between 0 and 2, can satisfy various videos through We use block matching criteria which is SAD, 3.

(4) our test. If β = 0 , the performance of MFHS is. Step 2.. If one of the suitable checkpoints is. similar to FHS. In this paper, we use fixed-valued of. located at the center of search window,. β to obtain a dynamic threshold value.. we build the ending pattern. Otherwise, the every checkpoint is set the center of new FHS, and every FHS adds three new. 3.3 Selection of search pattern. checkpoints. We find the minimum Because the adjustable multi-path search algorithm. BDM out from these new checkpoints,. has not restricted to the search pattern whose kinds, it. and judge whether the minimum BDM is. usually uses much simpler search pattern then much. less than SADmin . If yes, the minimum. easier realization. The matched search patterns are. BDM is defined as SADmin .We judge. the search patterns of 3SS, 4SS, DS, HEXBS and. whether the new checkpoints are suitable. FHS which we proposed. But the search pattern of. for eq. (2). If yes, we record the. CDS is difficult to apply due to the method of CDS is. corresponding locations if the suitable. complexity. The characteristics of original search. checkpoints and repeat step 2. If the. pattern and applied environment cause what kind of. method is not built a new FHS, we will. search patter is suitable for the adjustable multi-path. stop the algorithm. The location of. search algorithm. In solutions of QCIF, QSIF, CIF. SADmin is the motion vector.. and SIF, we suggest the search pattern of FHS to use because the size of the search pattern of FHS is small. Fig. 2 is the example of MFHS. We define T as. and suitable for lower solution. We suggest the search. 25. The motion vector is (4,-1) and we cost 24. pattern of DS to use in solution of CCIR601. In the. checkpoints to obtain the motion vector.. paper, we use mainly the search pattern for multi-path search algorithm because the number of checkpoints -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7. is less and the probability is more accurate than DS. 3.4 The adjustable multi-path flatted hexagon search algorithm The algorithm of the adjustable multi-path FHS (MFHS) is described as follows: Step 1.. The center of search window is starting point, and we calculate the 7 checkpoints of FHS. We find the minimum BDM out which is defined as SADmin . We judge whether the 7 checkpoints are suitable. -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7. 700. 900 600. 70. 100 800. 800 100. 75. 100. 800. 20. 800. 50. 100. 10. 200. 90. 40. 60. 100. 200. 80 200. First Step. Second Step. Third Step. Forth Step. for eq. (2). If yes, we record the corresponding locations of the suitable. Fig. 2 The searching example of multi-path. checkpoints, and go to Step 2.. flatted hexagon search algorithm. 4.

(5) has another feature which we add a little β to. 4. EXPERIMENTAL RESULTS. increase the accurate probability and the checkpoints are not added a lot.. A theoretical analysis about those fast BMAs has the. In Table 1, the feature of Salesman sequence is. implementation of such fast algorithms with several. highly centralized motion vector, and Coastguard. representative sequences of various motion contents. sequence is the smaller motion vector. MFHS is. can provide a realistic and interesting evaluation. For. higher probability and less checkpoints than the other. the purpose of comparison, six previous fast BMAs. methods on these sequences. When β = 0.5 , the. including 3SS, 4SS, N3SS, DS, HEXBS and CDS. probability and MAD of MFHS are close to FS, and. and the proposed MFHS algorithm (with various β ). they are 14.17 times and 11.33 times than the. are simulated by using the luminance component of. probability and MAD of FS, respectively. Fig. 4. five popular sequences: “Salesman” (CIF, 499. shows that the motion compensated results of the. frames), “Coastguard” (CIF, 300 frames), ”Garden”. 66th frame of the “Tennis” sequence with various β ,. (SIF, 115 frames), ”Tennis” (SIF, 67 frames) and. and we can see the phenomenon of the improvement. “Football” (SIF, 125 frames). Evaluation with such. is slowly increase in accurate rate. When β = 0.36 ,. five sequences in terms of MAD (mean absolute. the compensated frames of MFHS are similar to FS.. been. given. in. the. above. section,. but. distortion) used as the BDM, matching probability (i.e., the probability of finding the true motion vector). 5. CONCLUSIONS. and number of search points is described in Table 1. For each sequence, the search is performed at a block size of 16×16 within a window of size ±7. In Fig.. The advantages of MFHS are adjustable search speed. 3(a), we describe that the various β within MFHS. and matching probability. And, it can adaptive for. influenced the numbers of the checkpoints on five. various video contents and features. MFHS makes. sequences. In Fig. 3(b), we describe that the various. use of the adaptive threshold for keeping higher. β within MFHS influenced the accurate probability. probability on various image sequences. If you want. on five sequences. Fig. 4 shows the motion. to obtain the close probability to FS, you can set β. compensated results at various. β. on tennis. which is between 0.36 and 2 and MFHS is faster than FS. If you want to get faster speed, you can set β. sequence. In Fig. 3(a), we can know that the various β. which is below 0.22. But, we don’t suggest that you. within MFHS influenced the speed on various image. defined β as 0 because the smaller value of β. sequences. The correlation of β and the number of. can increase the accurate probability and reduce the. checkpoints is roughly linear on various image. search speed. MFHS is the higher probability method. sequences. Fig. 3(b) shows the correlation of β and. for the general methods. If you want to adjust the. probability. The general fast motion estimations are. speed and probability for various image sequences,. difficult to maintain the higher probability in. MFHS is what we are recommended.. complex motion vector. However, MFHS has a characteristic which can guarantee the probability in sufficient β . From Fig. 3(b) and Table 1, the MFHS. 5.

(6) 6. REFERENCES. [8] K. R. Rao and J. J. Hwang, Techniques and Standards for Image, Video, and Audio Coding.. [1] C. H. Cheung and L. M. Po,” A Novel. New Jersey: Prentice Hall, Inc., 1996.. Cross-Diamond Search Algorithm for Fast Trans.. [9] L. K. Liu and E. Feig, “A block-based gradient. Circuits Syst. Video Technol., vol. 12, No. 12,. descent search algorithm for block motion. Dec. 2002.. estimation in video coding,” IEEE Trans.. Block. Motion. Estimation”,. IEEE. Circuits Syst. Video Technol., vol. 6, pp.. [2] Ge Zhu, Xiao Lin, and Lap-Pui Chau,. 419–423, Aug. 1996.. “Hexagon-Based Search Pattern for Fast Block. [10] L. M. Po and W. C. Ma, “A novel four-step. Motion Estimation”, IEEE Trans. Circuits Syst. Video Technol., vol. 12, No. 5, May 2002.. search. algorithm. for. fast. block. motion. estimation,” IEEE Trans. Circuits Syst. Video. [3] Information Technology—Coding of Moving. Technol., vol. 6, pp. 313-317, June. 1996.. Pictures and Associated Audio for Digital Storage Media at up to About 1.5 Mbit/s–Part 2:. [11] R. Li, B. Zeng, and M. L. Liou, “A new. Video, ISO/IEC 11 172-2 (MPEG-1 Video),. three-step search algorithm for block motion. 1993.. estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 4, pp. 438-443, Aug. 1994.. [4] Information Technology—Generic Coding of Moving. Pictures. Information:. and. Video,. Associated. ISO/IEC. 13. Audio. [12] S. Zhu and K. K. Ma, “A new diamond search. 818-2. algorithm for fast block-matching motion. (MPEG-2 Video), 2000.. estimation,” IEEE Trans. Image Processing, vol. 9, pp. 287-290, Feb.2000.. [5] Information Technology—Coding of Audio. [13] T. H. Chen and Yi-Fan Li “A Novel Flatted. Visual Objects - Part 2: Visual, ISO/IEC 14. Hexagon Search Pattern for Fast Block Motion. 469-2 (MPEG-4 Visual), 1998.. Estimation”,. IEEE. 2004. International. Conference on Image Processing (ICIP),. [6] ITU-T. Recommendation. Singapore, Oct. 2004.(pp. 1477-1480). H.264/ISO/IEC. 11496-10, “Advanced Video Coding”, Final Committee. Draft,. Document. [14] T. H. Chen,” A Cost-Effective 3-Step. JVC-E022,. September 2002.. Hierarchical Search Block-Matching Chip for Motion. [7] J. Y. Tham, S. Ranganath, M. Ranganath, and A.. Estimation”,. IEEE. Journal. of. Solid-State Circuits, vol. 33., No. 8, pp. 1253-1258, Aug. 1998.. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motion. [15] T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T.. estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, pp. 369-377, Aug. 1998.. Ishiguro, “Motion compensated interframe coding for video conferencing,” in Proc. 6.

(7) National. Telecommunications. Conf.,. New. Orleans, LA, Nov. 1981, pp. G5.3.1-G5.3.5.. [16] Video Codec for Audiovisual Services at p × 64 kbits/s, ITU-T Recommendation H.261, Mar. 1993.. [17] Video. Coding. Communication,. for ITU-T. Low. Bit. Rate. Recommendation. H.263, Feb. 1998.. 7.

(8) Table 1 Simulation results of MFHS and other BMAs Tennis. Salesman BMAS. MAD. Pro.. Points. BMAS. MAD. Pro.. Points. FS. 2.775. 1.000. 204.283. FS. 5.249. 1.000. 202.048. 3SS. 2.825. 0.946. 23.212. 3SS. 6.211. 0.729. 23.111. N3SS. 2.781. 0.979. 16.776. N3SS. 5.656. 0.840. 20.293. 4SS. 2.817. 0.953. 16.144. 4SS. 5.659. 0.850. 18.455. DS. 2.814. 0.952. 12.892. DS. 5.453. 0.900. 15.943. HEXBS. 2.826. 0.943. 10.565. HEXBS. 5.842. 0.746. 12.651. CDS. 2.783. 0.976. 9.413. CDS. 5.487. 0.886. 15.065. MFHS(β). MAD. Pro.. Points. MFHS(β). MAD. Pr.. Points. 1. 2.775. 0.996. 23.914. 1. 5.267. 0.981. 69.430. 0.5. 2.776. 0.991. 14.413. 0.5. 5.290. 0.967. 36.904. 0.36. 2.778. 0.987. 12.789. 0.36. 5.319. 0.960. 29.262. 0.22. 2.779. 0.984. 11.912. 0.22. 5.385. 0.944. 22.340. 0.1. 2.780. 0.980. 11.417. 0.1. 5.502. 0.910. 16.610. 0.05. 2.782. 0.977. 11.078. 0.05. 5.585. 0.883. 14.660. FHS(0). 2.785. 0.966. 10.640. FHS(0). 5.718. 0.843. 13.093. Coastguard. Garden. BMAS. MAD. Pro.. Points. BMAS. MAD. Pro.. Points. FS. 4.657. 1.000. 204.283. FS. 8.856. 1.000. 202.048. 3SS. 4.753. 0.974. 23.375. 3SS. 9.853. 0.834. 23.204. N3SS. 4.698. 0.978. 19.786. N3SS. 9.005. 0.930. 21.139. 4SS. 4.726. 0.984. 18.500. 4SS. 9.463. 0.868. 18.696. DS. 4.711. 0.989. 16.509. DS. 9.108. 0.929. 16.638. HEXBS. 4.757. 0.967. 12.909. HEXBS. 9.745. 0.815. 13.048. CDS. 4.713. 0.988. 15.917. CDS. 9.046. 0.938. 14.998. MFHS(β). MAD. Pro.. Points. MFHS(β). MAD. Pro.. Points. 1. 4.658. 0.996. 23.914. 1. 8.859. 0.997. 38.988. 0.5. 4.662. 0.991. 14.413. 0.5. 8.870. 0.991. 26.175. 0.36. 4.666. 0.987. 12.789. 0.36. 8.882. 0.987. 22.583. 0.22. 4.674. 0.984. 11.912. 0.22. 8.912. 0.980. 18.651. 0.1. 4.694. 0.980. 11.417. 0.1. 8.945. 0.968. 15.253. 0.05. 4.713. 0.977. 11.078. 0.05. 8.978. 0.956. 14.115. FHS(0). 4.738. 0.966. 10.640. FHS(0). 9.040. 0.937. 13.281. 8.

(9) 120 100 Salesman. 80 Points. Coastguard 60. Tennis. 40. Garden Football. 20 0 0. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 β. (a). 1. 0.96. Probability. Salesman Coastguard. 0.92. Tennis 0.88. Garden Football. 0.84. 0.8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1. 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2. β. (b) Fig 3 (a) The correlation of the β and checkpoints. (b) The correlation of the β and probability.. 9.

(10) (b). (a). (c). (d). (e). (f). Fig4 A visual comparison using the SIF sequence “ Tennis” motion-compensated for: (a) the 66th frame of the original sequence (non-compensated); (b)FS, MSE =155.7; (c)MFHS, β=0, MSE=233.2; (d) MFHS, β=0.1, MSE=198.1; (e) MFHS, β=0.22, MSE=178.8; (f) MFHS, β=0.36, MSE=163.2;. 10.

(11)